FlowingData Forums » Tag: big data

FlowingData Forums » Tag: big data - Recent Posts http://forums.flowingdata.com/ Strength in Numbers en Sat, 11 Feb 2012 19:18:57 +0000 Data Storyteller at Knewton - NYC http://forums.flowingdata.com/topic/data-storyteller-at-knewton-nyc#post-2404 Fri, 18 Nov 2011 17:08:38 +0000 dzwieback 2404@http://forums.flowingdata.com/ Knewton is looking for master data storyteller to explore vast and growing quantities of educational data. You'll be joining an elite team that is building the world’s most powerful adaptive learning platform. You are obsessive about data visualization, a fearless defender of truth in data, and an experienced exterminator of chartjunk. You are a master or mistress of SQL, Excel, Tableau (and the like). You have some programming experience (python, perl, shell, etc.), and familiarity with statistical methods and tools (e.g., R). Above all, you are intellectually curious. Knewton is based in NYC’s Union Square, but you can work from pretty much anywhere. Benefits include Macs with huge monitors, a kitchen filled with your favorite treats, flexible hours, frequent guest speakers, company-sponsored training and conferences, excellent health benefits, equity, and a chance to do meaningful, cutting edge work. Apply at <a href="http://jobs.knewton.com/apply/u4UOuS/Master-Data-Storyteller.html" rel="nofollow">http://jobs.knewton.com/apply/u4UOuS/Master-Data-Storyteller.html</a> . Pythonic Software Engineer http://forums.flowingdata.com/topic/pythonic-software-engineer#post-2338 Tue, 13 Sep 2011 02:46:24 +0000 josverwoerd 2338@http://forums.flowingdata.com/ Job Description We seek a PEP8 Software Engineer with 5+ years experience developing large-scale, distributed applications. BigML's development is done in Corvallis, OR. Responsibilities: Develop new highly scalable intelligent applications capable of processing hundreds of terabytes of data. Desired Skills & Experience Must Have: BS degree in Computer Science. 5 years of large-scale software design and development experience, with extensive knowledge of Unix. 4 years experience with C/Python and PyPy. Best software engineering practices. Passionate about coding. Passionate about distributed applications. Passionate about machine learning applications. Passionate about solving large-scale problems. Nice to have: Experience with MongoDB, replica sets, and sharding. Experience with HBase and Hadoop. Experience with RabbitMQ. Experience with Tornado. Experience with Django. Experience with Nginx. Masters or Ph.D. in Computer Science. Company Description BigML, Inc is a new start-up that is developing a highly-scalable, easy-to-use machine learning service for big data. We are seeking world-class software engineers who are passionate about creating the next generation of intelligent services. If you want to change the world, try helping us predict its future. Data-Visualization/UI Lead Developer for Venture-Backed Startup (Quantifind) http://forums.flowingdata.com/topic/data-visualizationui-lead-developer-for-venture-backed-startup-quantifind#post-1880 Tue, 07 Sep 2010 19:31:37 +0000 quantifind 1880@http://forums.flowingdata.com/ Title: Data-Visualization/UI Lead Developer for Venture-Backed Startup (Quantifind) Company: Quantifind, Inc. Location: Palo Alto, CA --- Description: Quantifind is a well-funded startup located in Palo Alto, CA, supported by a tier-1 VC and founded by physicists from Stanford University. We have developed a powerful platform for unstructured-data analytics and are seeking an experienced User Interface virtuoso to unleash its full potential. The ideal candidate should be passionate about data visualization and capable of designing compelling, exploratory visualizations of data dynamically extracted from huge unstructured repositories. Our product has already elicited from early customers the feeling of power that the best big-data applications bring. The candidate's job will be to combine this processing potential with the sense of front-end magic that the best modern visualization tools provide. With a combination of programming skills, an intuitive sense of design, and an ability to integrate user feedback, we expect the applicant to create a polished product that resonates with the customer's need to process information efficiently, collectively, and visually. We are most interested in seeing a portfolio that demonstrates both originality and an awareness of usability from an applicant, although several years of experience is a bonus. This lead developer position comes with a very competitive compensation and equity offer that matches the large impact that the position is expected to provide. --- Recommended skills: - UI, UX, HCI - Javascript, jQuery - HTML5, CSS, Canvas, SVG - Web-based plotting/charting visualization libraries - R, Processing, Tableau - Google Web Toolkit (GWT) - Java - Grails/Rails - Adobe Illustrator, Photoshop --- Experience with... …modern web-based data analysis and visualization tools …building visual (or standard) Information Retrieval/Search interfaces …design; aesthetic principles concerning the communication of quantitative data (Tufte) …big-data science and general methods for large-scale statistical processing of scientific data …user testing, both direct and tracking --- Quantifind welcomes applicants of all origins and will sponsor H1B candidates. --- Contact: Please email <a href="mailto:careers@quantifind.com" rel="nofollow">careers@quantifind.com</a> with a resume, portfolio, or questions regarding this opening. (Resumes using Comic Sans should be sent to <a href="mailto:omgwtf@quantifind.com" rel="nofollow">omgwtf@quantifind.com</a> for expedited processing.) MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-686 Wed, 06 May 2009 04:01:01 +0000 java-nokia 686@http://forums.flowingdata.com/ Hello, Very Nice forum! <a href="http://xn--90aeflbqelf1a1jeg.com/" title="Java Nokia" rel="nofollow">Java for Nokia</a> MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-386 Fri, 06 Feb 2009 17:44:55 +0000 dhruv 386@http://forums.flowingdata.com/ machetEC2 is released! Check out our <a href="http://machetec2.org" rel="nofollow">blog post</a> (soon to be wiki) for more details. The id of the image is ami-29ef0840. Launch it and try it out! Digging Into Data Challenege http://forums.flowingdata.com/topic/digging-into-data-challenege#post-382 Thu, 05 Feb 2009 17:08:30 +0000 cloister 382@http://forums.flowingdata.com/ It means that research changes focus. The old focus was on discovering new facts. The new focus is on discovering patterns of facts. This is particularly notable in archaeology. If ever there was a science whose public image is on the literal discovery of new facts--in this case, arti"facts" buried within historical sites--archaeology would have to be it. But in the past decade or so, archaeologists have begun to put all their data into databases and to data-mine them. In the American Southwest, for example, there are now databases that are composed of the ages, locations, and many other parameters of thousands of dig sites and millions of objects from the American Indian cultures that previously lived there. And data mining is starting to provide answers to questions that were previously unapproachable. Who were these people? What were their geographical ranges? How did they trade goods between each other, often across vast distances, on foot? How did one society's art and technology affect its neighbors? By looking at patterns in the data, such as projectile points vs. knapping style vs. time, researchers can get a picture of where a particular technology originated, when it originated, and how it spread across the Southwest and Central America. Yes, discovering a new dig site can be exciting and revolutionary. But in another sense, discovering yet another dig site is kind of ho-hum. After you've already got a few million of them catalogued, digging up another few hundred arrowheads or broken pots or whatever becomes kind of meaningless. It's only when you can put that new data into context with all the rest and look at the patterns that emerge, can the data take on any real meaning. Digging Into Data Challenege http://forums.flowingdata.com/topic/digging-into-data-challenege#post-381 Thu, 05 Feb 2009 00:41:40 +0000 nathany 381@http://forums.flowingdata.com/ <a href="http://www.diggingintodata.org/" rel="nofollow">http://www.diggingintodata.org/</a> [via <a href="http://delicious.com/keyvowel" rel="nofollow">keyvowel</a>] <blockquote>What is the "challenge" we speak of? The idea behind the Digging into Data Challenge is to answer the question "what do you do with a million books?" Or a million pages of newspaper? Or a million photographs of artwork? That is, how does the notion of scale affect humanities and social science research? Now that scholars have access to huge repositories of digitized data -- far more than they could read in a lifetime -- what does that mean for research?</blockquote> MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-336 Wed, 28 Jan 2009 23:18:23 +0000 nathany 336@http://forums.flowingdata.com/ Ah, I get it now MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-334 Wed, 28 Jan 2009 19:35:43 +0000 dhruv 334@http://forums.flowingdata.com/ Amazon's <a href="http://aws.amazon.com/ec2/" rel="nofollow">EC2</a> allows users to instantiate a virtual computer with a pre-installed operating system, software packages, and up to 1 TB of data pre-loaded on disk, ready to work with, from a pre-defined image (an "Amazon Machine Image", or AMI). MachetEC2 is an effort by a group of <a href="http://infochimps.org" rel="nofollow">Infochimps</a> to create an AMI for data processing, analysis, and visualization. If you create an instance of MachetEC2, you'll be have an environment with tools designed for working with data ready to go. All you'll need to do is load in some data and get to work! MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-332 Wed, 28 Jan 2009 18:43:49 +0000 nathany 332@http://forums.flowingdata.com/ So in layman's terms... MachetEC2 would be used to make data storage and retrieval easier with Amazon EC2? MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-331 Wed, 28 Jan 2009 18:40:20 +0000 mrflip 331@http://forums.flowingdata.com/ Wow, 15 minutes later and already some great additions so far from <a href="http://search.twitter.com/search?q=+infochimp+OR+infochimps+OR+machetec2" rel="nofollow">twitter</a>. Including: * @hackingdata: prefuse/flare * @peteskomoroch: @infochimps I did something similar a while back, might want to add ipython, boto. some libs I considered: <a href="http://tinyurl.com/ckl2zr" rel="nofollow">http://tinyurl.com/ckl2zr</a> (<-- whoa. Awesome.) * @dwf: Mayavi2 and its dependency, VTK: Great for 3D viz stuff. PyMC, definitely. matplotlib. Maybe Boost? * @grantmichaels: erlang * @ogrisel: happy or disco for mapreduce, numpy, scipy, pylab, boto, mdp, lxml, liblinear, cython, gcc, torch5 Already more than our finite monkeys can roll out, but with this kind of interest we'll get the ball rolling (or get behind @peteskomoroch's ball and help push) and watch what happens from there. MachetEC2: Open Visualization / Big Data Toolkit on Amazon EC2 http://forums.flowingdata.com/topic/machetec2-open-visualization-big-data-toolkit-on-amazon-ec2#post-330 Wed, 28 Jan 2009 17:58:39 +0000 mrflip 330@http://forums.flowingdata.com/ Hi, We're assembling MachetEC2, a free Amazon AWS EC2 instance that comes out of the box with a full suite of data analysis and visualization tools. (Obviously the compute time is on your dime, but the base instance is there for you to grab, for free, and get to hackin') What tools do you want to have in there? We're going to push this up in the next few days -- look for an announcement on the <a href="blog.infochimps.org" rel="nofollow">infochimps blog</a>, and we'll reply here too. If you'd like to help, set up something you find essential and tweet/email us to 'pull' from your instance. (You'll have to use a credit card to get an instance, but it's so cheap it'll only cost a buck or two; and once it's set up we'll pull it onto our account where everyone can get it.) If there's already something like this floating around, please let us know and we'll work with them instead. We'll initially post up some subset of: * Ruby, Python, Erlang, R * MySQL, PostgreSQL * AllegroGraph, CouchDB * Hadoop, Hive, Pig * Cytoscape, Gruff * Processing, Prefuse/Flare, Modest Maps * NLTK, SciPy ... * YOUR SUGGESTION HERE This will fit in with the things we're helping to add to the AWS Public Data Sets. We want you to be able to use infochimps to find something or pull down the Public Data Set of your choice; to load that dataset next to your MachetEC2 instance clone, and be whailing on data from go. At some later point we can specialize (MachetEC2-viz, MachetEC2-ML for machine learning folks, MachetEC2-ling for linguistics, etc), so feel free to suggest things of (somewhat) narrow focus, and even more free as I said to set it up and ping us. Free/Open software only, of cours.