FlowingData Forums » Tag: software

FlowingData Forums » Tag: software - Recent Posts http://forums.flowingdata.com/ Strength in Numbers en Sat, 11 Feb 2012 19:22:35 +0000 Using gapminder like they do in the video! http://forums.flowingdata.com/topic/using-gapminder-like-they-do-in-the-video#post-2501 Fri, 20 Jan 2012 08:08:54 +0000 frankc 2501@http://forums.flowingdata.com/ A bit late to this post, but StatPlanet Plus enables you to do this, i.e. import your own sub-national data and create motion bubble charts (free desktop version for personal / non-commercial use) Where do I start? http://forums.flowingdata.com/topic/where-do-i-start#post-2240 Wed, 22 Jun 2011 12:54:34 +0000 cscott 2240@http://forums.flowingdata.com/ Thank you for the information, nathany. It is very helpful. Where do I start? http://forums.flowingdata.com/topic/where-do-i-start#post-2239 Tue, 21 Jun 2011 13:41:24 +0000 nathany 2239@http://forums.flowingdata.com/ For online reports of public data, Tableau Public could be a good place to start. However, I have a hunch your clients will want to keep your data private, which means you'll need to look elsewhere and possibly build your own reports. For that the FlowingData book might be a good start (<a href="http://book.flowingdata.com" rel="nofollow">http://book.flowingdata.com</a> :). As for question 2, here's what I told someone else with a similar question: <blockquote>The first thing I'd look at after the portfolio would be their experience in handling data. Most programmers can make something with numbers, but those who can get more from just the straight up digits from the Excel spreadsheet will give you better results. I'm thinking relationships between variables like trends, differences, correlations, outliers, etc. Since interactives are meant to let users explore their own questions, the designer has to know the right questions to ask about the data. If you wanted to be thorough about it, I'd probably give them a few graphics, and ask what they thought about them... good, bad, improvements... general curiosity about data is of course good. As far as core competencies, experience with Actionscript and JavaScript are pretty standard. Probably should already know about JS packages like D3, Protovis, and Processing.js. Experience with R would be a plus, not so much for the interaction but for the analysis beforehand (which is most important, I think).</blockquote> hth. Where do I start? http://forums.flowingdata.com/topic/where-do-i-start#post-2238 Tue, 21 Jun 2011 09:56:28 +0000 cscott 2238@http://forums.flowingdata.com/ Hi, I'm an in-house print/UX designer for a research firm. We are looking to package our data for clients in the form of online reports containing interactive graphs/charts and state maps that would need to illustrate the state’s counties, cities, and economic regions. My experience with data visualization has been limited to cutting data out of an Excel file and pasting it into Illustrator to recreate a stylized graphic for print. This leaves me with many questions, but at this point I am working on these two: 1. Would a system like Tableau fulfill the project requirements listed above? Are there obvious limitations to the types of visualizations you can create and who you can share them with? 2. If we determined we had the budget to build a "data visualization team", what type of person (or people) should we look for and what type of skill set would they need? In my mind this is someone who can pull data with code and create custom Flash animations or something similar, but I’m not sure if there is an official title for this or even the best method. I’ve read the 40 Essential Tools article and found it to very helpful. Any additional guidance will be appreciated. Thanks! Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-1978 Mon, 08 Nov 2010 20:01:02 +0000 markslay 1978@http://forums.flowingdata.com/ more of a substantive thought: IMO, it depends on what you are really trying to go for. If you want to do more cool applets with complex, intelligent, object-oriented programming, then Processing is much better, but if you're designing for consumer-based (commerce-focused) interaction, then flash is the answer. Art = Processing, Advertising = Flash, more simply put... in my humble opinion. For example, see this cool applet "LifeForms" which simulates an ecosystem: <a href="http://web.ics.purdue.edu/~omalled/omalled.com/Applets.html" rel="nofollow">http://web.ics.purdue.edu/~omalled/omalled.com/Applets.html</a> Factual looking for 3 software engineers, 1 data engineer http://forums.flowingdata.com/topic/factual-looking-for-3-software-engineers-1-data-engineer#post-1784 Fri, 18 Jun 2010 14:42:07 +0000 gnestor 1784@http://forums.flowingdata.com/ Factual is a well-funded and expanding start-up with a mission to create a platform where anyone can share and mash open data on any subject. We are pioneers in the worldwide trend towards open information flow, and our goal is to provide the infrastructure to curate high-quality data and make it accessible everywhere. See our Jobs page for details: <a href="http://www.factual.com/jobs" rel="nofollow">http://www.factual.com/jobs</a> Position: Data Software Engineer Location: Los Angeles, CA Description: Factual is looking for an engineer with good attention to detail that has both experience with and passion for large data. The candidate would take ownership of our data pipeline and manage the process and schedule to ensure timely delivery of data. The ideal candidate would: Have a CS degree Have 2+ years of professional hands-on software experience Have some experience with cloud computing or cluster management Be enthusiastic about large data Be eager to learn new tools and skills related to data and scripting Be able to propose out-of-the-box strategies based on insights about data Desired Skills (at least three of the following): Proficient in a language such as C++, Java, Python, Ruby, Perl Linux/bash scripting SQL and/or Extract/Transform/Load (ETL) frameworks HTML DOM/XPath and/or Regular expressions Project management Responsibilities: Oversee entire data pipeline and schedule Evaluate and judge algorithmically generated data and assess data quality Coordinate and run web-scale jobs on large clusters and write automation scripts Aggregate, clean, and merge data Generate, maintain, and operate the generation software and scripts Author CSS and XPath selectors Position: Data Engineer Location: Los Angeles, CA Description: Factual is looking for a fast-working multi-tasker with good attention to detail that has both experience with and passion for large data. The ideal candidate would: Have a degree in engineering, science, or linguistics Be eager to learn new tools and skills related to data and scripting Be able to propose out-of-the-box strategies based on insights about data Desired Skills (at least three of the following): Proficient in a language such as C++, Java, Python, Ruby, Perl Linux/bash scripting SQL and/or Extract/Transform/Load (ETL) frameworks HTML DOM/XPath and/or Regular expressions Speaks multiple languages Experience with data curation Linguistics research Project management Responsibilities: Research and analyze data on the web Evaluate and judge algorithmically generated data and assess data quality Curate data schemas Aggregate, clean, and merge data Generate, maintain, and operate the generation software and scripts Find and follow URL patterns using regular expressions Author CSS and XPath selectors Write automation scripts Position: Software Engineer, Large Data Processing Location: Los Angeles, CA Description: Factual Inc. has an immediate need for an experienced full-time software engineer who is excited about algorithms and large-scale data processing. We need a strong new team member for our mission to develop groundbreaking systems to cleanse huge datasets on-the-fly. If you are a seasoned software engineer who loves data and knows how to beat the averages, we'd like to talk to you. The ideal candidate would: Generally love data Be familiar with MapReduce programming models such as Hadoop Have good knowledge of parsing, duplicate detection, statistical and clustering algorithms Thrive in a start-up environment and enjoy wearing many hats Desired Skills: Degree in Computer Science or other technical / scientific field Fluency in Java Strong algorithms and data structures knowledge Experience in writing fast production code and code optimization Familiarity with databases, especially NoSQL databases Comfort with Unix and the command line Responsibilities: Collaborate with both the data team and the back-end team to design and develop our core production real-time data classification and improvement algorithms Create metrics, reports, and management tools to measure the effectiveness of our data cleansing - then use this feedback to further refine our next-generation algorithms Shepherd your product from initial concept to hands-on development, testing, and then all the way to deployment and production monitoring Position: Software Engineer, NoSQL Specialist Location: Los Angeles, CA Description: Factual Inc. has a pressing need for an experienced full-time software engineer who has a great fascination and knowledge of the NoSQL space - systems like HBase and Cassandra. Our company is all about data and we need to store and retrieve huge datasets with a guarantee of high availability and performance. Our ideal new team member is excited about working for a start-up and not afraid of a fast development cycle. The ideal candidate would: Generally love data Have a strong interest in cluster computing Thrive in a start-up environment and like wearing many hats Desired Skills: Degree in Computer Science or other technical / scientific field Fluency in Java Deep knowledge of BigTable, Cassandra, HBase, or a similar NoSQL database in production (experience with multiple NoSQL databases a big plus!) Experience in writing fast production code and code optimization Comfort with Unix and the command line EC2 experience a plus Responsibilities: Work on our back-end server team to design and develop a data storage platform that can serve the world Tune and tweak performance, from algorithms to machine configuration Shepherd your product from initial concept to hands-on development, testing, and then all the way to deployment and production monitoring Looking for software http://forums.flowingdata.com/topic/looking-for-software#post-1527 Mon, 15 Feb 2010 10:59:44 +0000 nathany 1527@http://forums.flowingdata.com/ i've heard a lot of good things about <a href="http://tableausoftware.com/flowingdata" rel="nofollow">tableau software</a> (disclosure: they're an FD sponsor) Looking for software http://forums.flowingdata.com/topic/looking-for-software#post-1525 Mon, 15 Feb 2010 10:49:54 +0000 mikev 1525@http://forums.flowingdata.com/ Does anyone know of any comprehensive data visualization software packages? Something that can do all of the things that the ManyEyes website can do would be ideal. Using gapminder like they do in the video! http://forums.flowingdata.com/topic/using-gapminder-like-they-do-in-the-video#post-1519 Fri, 12 Feb 2010 07:38:27 +0000 Anonymous 1519@http://forums.flowingdata.com/ Trend Compass does what you need, which is "Drilling data". Ossama Using gapminder like they do in the video! http://forums.flowingdata.com/topic/using-gapminder-like-they-do-in-the-video#post-1500 Sat, 06 Feb 2010 23:26:53 +0000 nathany 1500@http://forums.flowingdata.com/ Happy to hear it, dc321 :). Yes, those posts are still best I advice to give. For scraping, I use Python, but I know a lot of people really like Ruby too. However, if you're already comfortable with Perl, I say just use that. Visualization-wise, those might not be as helpful. For that, go with Flash/Actionscript or Processing, or even javascript. This is all assuming you're Web-based... Using gapminder like they do in the video! http://forums.flowingdata.com/topic/using-gapminder-like-they-do-in-the-video#post-1491 Fri, 05 Feb 2010 11:17:15 +0000 dc321 1491@http://forums.flowingdata.com/ Hi Nathan - Thanks so much for the reply. If I was interested in creating or tweaking graphics like that, what do you think would be the best softwares to look into? Also, do you have any suggestions of what would be good programming languages to use? I used PERL some years back to do webscraping and got fairly technical, but have been told that Ruby is probably a better language to learn these days for that purpose. Would Ruby be helpful for this? More generally, do the below threads still represent your best advice for what languages/tools would be most useful to learn in order to create data visualizations? <a href="http://flowingdata.com/2009/09/03/what-visualization-toolsoftware-should-you-use-getting-started/" rel="nofollow">http://flowingdata.com/2009/09/03/what-visualization-toolsoftware-should-you-use-getting-started/</a> <a href="http://flowingdata.com/2008/10/20/40-essential-tools-and-resources-to-visualize-data/" rel="nofollow">http://flowingdata.com/2008/10/20/40-essential-tools-and-resources-to-visualize-data/</a> Thanks so much. I really appreciate all the work you put into FlowingData. Definitely one of my favorite sites! Using gapminder like they do in the video! http://forums.flowingdata.com/topic/using-gapminder-like-they-do-in-the-video#post-1484 Thu, 04 Feb 2010 11:24:40 +0000 nathany 1484@http://forums.flowingdata.com/ nothing comes to mind, unfortunately. i think you'd have to go custom. Using gapminder like they do in the video! http://forums.flowingdata.com/topic/using-gapminder-like-they-do-in-the-video#post-1480 Wed, 03 Feb 2010 20:20:04 +0000 dc321 1480@http://forums.flowingdata.com/ Hello all - I recently started playing with Google Chart's Motion chart which is the same as Gapminder. However, I was wondering if there is a way to look WITHIN a country. For example, Gapminder has millions of year-country specific data. At some point in the video, Hans Rosling clicks on a country and 5 smaller bubbles come out of a country showing the quintiles within a country. Is there a similar way to work with our own data set so that if I want to include subpopulations within a country or other unit that I can do this? I am also open to using other programs that will have the same functionality. Thanks so much! Software used http://forums.flowingdata.com/topic/software-used#post-1031 Mon, 03 Aug 2009 14:08:16 +0000 amenity 1031@http://forums.flowingdata.com/ Out of curiousity, has anyone tried Python for interactive visualization? E.G. <a href="http://travisvaught.blogspot.com/2009/08/infographic-in-python-using-chaco.html#comments" rel="nofollow">industrial production infographic</a> based on a NYTimes recession visualization? Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-733 Thu, 11 Jun 2009 06:54:49 +0000 kevincannon 733@http://forums.flowingdata.com/ I'd say go with Flash. You can get a discounter student version of it from Adobe. Flash is generally much easier to get to grips with because you can easily create objects and move them around just telling them to move from A to B. in Processing you need to worry about the maths behind it. Also, basic interactive features like buttons and dragging are built in to Flash whereas with Processing you need to code them each time. This is not difficult for someone who's experienced, but it's hassle for a beginner. If you're learning to program from scratch then you won't lose anything by dabbling with Processing, it's good for a lot of other things and the price is right, but when it comes to deploying things on the web, and interactivity, Flash wins hands down. Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-731 Thu, 11 Jun 2009 06:24:02 +0000 goldfisch 731@http://forums.flowingdata.com/ @femion try flash with json instead of xml. you will find an as2 class here <a href="http://www.json.org/json.as" rel="nofollow">http://www.json.org/json.as</a> i find it much easier to work with json, because the data structure can be accessed as object. Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-722 Tue, 09 Jun 2009 23:49:39 +0000 yghitza 722@http://forums.flowingdata.com/ Great, thanks very much, this is really helpful. Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-719 Tue, 09 Jun 2009 13:34:42 +0000 femion 719@http://forums.flowingdata.com/ I've done web-based science data visualizations in both Processing and in Flash. I haven't used Flex, and I programmed in AS 2.0. Flash Disadvantages: - data must be in xml and the way to search through your xml data was a little infuriating (at least in AS 2.0) - the geometric functions were a little limited (at least in AS 2.0). I think they've expanded a bit in AS 3.0. - proprietaryness. There is an open source AS compiler, but I never got it to work correctly. Flash Advantages: - timeline control - vector graphics editor built-in - easily compiled and exported to the web Processing disadvantages: - has some bugs when you try to import external libraries - has some bugs when you re-export your applets - applets must be signed if you're going to talk to the internet for anything, or else they'll crash. - no graphics editor - external images must be imported as vector graphics - no timeline editor (although the play functions are intuitive) Processing advantages: - not proprietary - can accept any type of data (not just xml) - nice tight knit community - lots of geometric functions I lean towards Processing. Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-718 Tue, 09 Jun 2009 13:05:03 +0000 nathany 718@http://forums.flowingdata.com/ if you're after interactive web viz, then processing and flash/actionscript are definitely where it's at. rosling's trendalyzer, for example, is in flash. Some people are using Flex too, which in the end comes down to Flash. I've done some stuff in Processing, which seems to take less time to actually start producing stuff if you're new to the area. People are doing really advanced stuff with Processing too though, so it's not just a learning tool. I think that since you're just beginning, you should go with Processing to start and move to Flash if it doesn't suit your needs. There's a good number of tutorials. Personally, I code in actionscript in Flex Builder. Students can get a free license here: <a href="https://freeriatools.adobe.com/flex/" rel="nofollow">https://freeriatools.adobe.com/flex/</a> The main thing is just getting started. I guess that's always the hardest part. Basically, if you learn either in depth, you'll always be able to go to the other later on, and it won't be a hard transition. Processing vs Flash? http://forums.flowingdata.com/topic/processing-vs-flash#post-716 Tue, 09 Jun 2009 11:58:53 +0000 yghitza 716@http://forums.flowingdata.com/ I'm new to advanced visualization tools (i.e. anything besides R and Excel) and trying to figure out what tools to learn. I'm hoping to make some custom interactive web-accessible visualizations -- think of Hans Rosling as a starting point -- and have some coding background. I get the sense that Processing and Flash/Actionscript are two of the main options. From what I can tell, people seem to describe Processing as a learning tool and a good place to start, but then you might want to move to something else. My question: what does Processing lack that Flash/Actionscript has? In other words, if I learn Processing (which is open source and has a ton of easily accessible learning materials), what are the drawbacks? Can Flash more easily handle large-scale data, advanced GUIs, something else? Is there something else I should consider? I looked around a little bit but haven't found satisfying answers to these questions. For what it's worth, I'm a student so cheap or free is always a bonus. At the same time, I'll be investing a lot of time so I'm not just looking for a temporary solution. Thanks for any help!