FlowingData Forums » Forum: Statistics and Data

FlowingData Forums » Forum: Statistics and Data - Recent Topics http://forums.flowingdata.com/ Strength in Numbers en Sat, 11 Feb 2012 22:04:14 +0000 Getting Started w/ your.flowingdata http://forums.flowingdata.com/topic/getting-started-w-yourflowingdata#post-2525 Wed, 08 Feb 2012 17:54:59 +0000 eklypse 2525@http://forums.flowingdata.com/ I'd like to use yfd on a protected account. That is, anyone who wants to follow me must submit a request which I can then approve. I hoped that whatever automated feature that would cause yfd to follow back would thus generate such a request which I could then approve, but as yet, no request has appeared. I started following @yfd a couple hours ago, so I figured the request would appear by now. Is there anything I can do to make this work (short of turning off the privacy on my account)? Thanks in advance! Looking for a Starbucks calculator http://forums.flowingdata.com/topic/looking-for-a-starbucks-calculator#post-2468 Wed, 04 Jan 2012 07:25:13 +0000 brcire 2468@http://forums.flowingdata.com/ In search of a Starbucks calculator that shows/graphs how much you spend per week, month, per year(s) and runs a parallel result of how that money would look if you invested it at standard interest rates. I recall one several years ago but cannot find it. Thanks for your help! Eric Where to find worldwide sunrise and sunset data set? http://forums.flowingdata.com/topic/where-to-find-worldwide-sunrise-and-sunset-data-set#post-2459 Thu, 29 Dec 2011 23:22:54 +0000 patrickberkeley 2459@http://forums.flowingdata.com/ Hi, does anyone know where I could find an open data set containing worldwide sunrise and sunset times? Thanks, Patrick Available: CRS, Health Insurance Coverage by State and Congressional District http://forums.flowingdata.com/topic/available-crs-health-insurance-coverage-by-state-and-congressional-district#post-2390 Mon, 24 Oct 2011 19:26:26 +0000 prescottrjp 2390@http://forums.flowingdata.com/ Available: <a href="http://goo.gl/WgKu5" rel="nofollow">http://goo.gl/WgKu5</a> Congressional Research Service, Health Insurance Coverage by State and Congressional District; <a href="http://goo.gl/WgKu5" rel="nofollow">http://goo.gl/WgKu5</a> Summary The total U.S. civilian non-institutionalized population in 2010 was estimated to be slightly more than 304 million. Roughly 84.5% of the U.S. civilian non-institutionalized population had one or more forms of health insurance, while 15.5%, or roughly 47.2 million, were uninsured. The most common form of insurance was employer provided. This report employs the U.S. Census Bureau’s 2010 American Community Survey (ACS) to describe health insurance coverage and provide estimates of coverage by type of coverage at the national, state, and congressional district level. The ACS survey has a sample of more than 2 million respondents and solicits health insurance coverage information as of the date of the survey. The sample is large enough to provide accurate estimates of coverage at the congressional district level at a point in time. As this report details, there are considerable differences across states, within states, and across demographic groups in the proportion of insured and their sources of coverage. Creating matrix for heatmap in R http://forums.flowingdata.com/topic/creating-matrix-for-heatmap-in-r#post-2339 Tue, 13 Sep 2011 04:27:54 +0000 harald 2339@http://forums.flowingdata.com/ I followed the excellent FlowingData tutorial "How to Make a Heatmap – a Quick and Easy Solution": <a href="http://flowingdata.com/2010/01/21/how-to-make-a-heatmap-a-quick-and-easy-solution/" rel="nofollow">http://flowingdata.com/2010/01/21/how-to-make-a-heatmap-a-quick-and-easy-solution/</a> In the tutorial, the baseball dataset was already prepared as a matrix with players as rows and columns with various points. What is the easiest way of creating a dataset like the baseball dataset <a href="http://datasets.flowingdata.com/ppg2008.csv" rel="nofollow">http://datasets.flowingdata.com/ppg2008.csv</a> in R if the data has the format: Dwyane Wade, G, 79 Dwyane Wade, MIN, 38.6 Dwyane Wade, PTS, 30.2 Dwyane Wade, FGM, 10.8 Dwyane Wade, FGA, 22 Dwyane Wade, FGP, 0.491 Dwyane Wade, FTM, 7.5 Dwyane Wade, FTA, 9.8 Dwyane Wade, FTP, 0.765 Dwyane Wade, 3PM, 1.1 Dwyane Wade, 3PA, 3.5 Dwyane Wade, 3PP, 0.317 Dwyane Wade, ORB, 1.1 Dwyane Wade, DRB, 3.9 Dwyane Wade, TRB, 5 Dwyane Wade, AST, 7.5 Dwyane Wade, STL, 2.2 Dwyane Wade, BLK, 1.3 Dwyane Wade, TO, 3.4 Dwyane Wade, PF , 2.3 LeBron James, G, 81 LeBron James, MIN, 37.7 LeBron James, PTS, 28.4 LeBron James, FGM, 9.7 LeBron James, FGA, 19.9 LeBron James, FGP, 0.489 LeBron James, FTM, 7.3 LeBron James, FTA, 9.4 LeBron James, FTP, 0.78 LeBron James, 3PM, 1.6 LeBron James, 3PA, 4.7 LeBron James, 3PP, 0.344 LeBron James, ORB, 1.3 LeBron James, DRB, 6.3 LeBron James, TRB, 7.6 LeBron James, AST, 7.2 LeBron James, STL, 1.7 LeBron James, BLK, 1.1 LeBron James, TO, 3 LeBron James, PF, 1.7 ....Kobe Bryant ...cont.... ...cont.... Doing such a transformation is straightforward in SQL or even a spreadsheet, but it would be great to know how to get it right in R. looking for a readable management (?) blog http://forums.flowingdata.com/topic/looking-for-a-readable-management-blog#post-2358 Mon, 26 Sep 2011 18:12:35 +0000 justin_abraham 2358@http://forums.flowingdata.com/ I am a mid-level manager at a Fortune 500 company with some time on my hands this quarter (don't ask). I am trying to broaden my horizons online, but everything I find is either: a) really interesting analytics/visualization stuff, which is sadly not my job b) disgruntled developers channeling Dilbert c) management or IT consultants trying to sell their wares with unreadable recycled jargon Will someone please recommend something as engaging, simple, and thought provoking as this site, but geared towards, say data warehousing or ETL? or even management? or are those topics just inherently boring (to write about)? Android app / widget? http://forums.flowingdata.com/topic/android-app-widget#post-2298 Wed, 10 Aug 2011 05:00:15 +0000 snataas 2298@http://forums.flowingdata.com/ Does anyone know of plans for an Android app, ideally with a widget for simple entry into your.flowingdata.com? I know there is one on github but it seems not to be available in the market. Thanks. Developer Contest with Infochimps data http://forums.flowingdata.com/topic/developer-contest-with-infochimps-data#post-2250 Tue, 05 Jul 2011 10:38:58 +0000 nathany 2250@http://forums.flowingdata.com/ Build a quick app with Infochimps data and win the Tufte books: <a href="http://blog.infochimps.com/2011/07/04/new-developer-contest-twilio-infochimps/" rel="nofollow">http://blog.infochimps.com/2011/07/04/new-developer-contest-twilio-infochimps/</a> 2010 Census Data got me down http://forums.flowingdata.com/topic/2010-census-data-got-me-down#post-2230 Tue, 14 Jun 2011 12:47:04 +0000 joemurphy 2230@http://forums.flowingdata.com/ All I want is to get average age by zip/tract but I'm lost. I've tried getting the 2010 census demogrphic profiles into access, but was left wanting to throw my computer out the window. Advice? <a href="http://2010.census.gov/2010census/data/" rel="nofollow">http://2010.census.gov/2010census/data/</a> Medicare Claims Data Challenge http://forums.flowingdata.com/topic/medicare-claims-data-challenge#post-2220 Fri, 03 Jun 2011 01:17:23 +0000 nathany 2220@http://forums.flowingdata.com/ Looks like it might be a good one: <a href="http://www.health2challenge.org/2011/06/01/medicare-claims-data/" rel="nofollow">http://www.health2challenge.org/2011/06/01/medicare-claims-data/</a> <blockquote>IMPAQ International (IMPAQ) and NORC at the University of Chicago wish to promote the development of internet-based applications for users of the newly released Medicare Claims Public Use Files. IMPAQ and NORC, with support from Centers for Medicare and Medicaid Services (CMS), seek to develop web platforms suited for health services researchers, health policy researchers, data entrepreneurs, and/or other potential users who would like to access Medicare claims data. This online event challenges interdisciplinary teams to create a tool for researchers that will present two or more of the new Basic Stand Alone (BSA) Medicare Claims Public Use Files (PFUs) in a way that’s informative, interactive, and user friendly.</blockquote> Prize: $7,500 and 2 tickets to Health 2.0 conference. Deadline: August 15 Best graph/visualization for travel time reliability? http://forums.flowingdata.com/topic/best-graphvisualization-for-travel-time-reliability#post-2195 Mon, 02 May 2011 13:07:42 +0000 LisaSmith 2195@http://forums.flowingdata.com/ I've been tasked with creating a "dashboard" for performance measures. In the past we've just used Excel charts. Does anyone have an interesting way to demonstrate reliability using Excel and/or Xcelsius? Gas Prices to Energy Company Stock Prices http://forums.flowingdata.com/topic/gas-prices-to-energy-company-stock-prices#post-2194 Sun, 01 May 2011 15:35:22 +0000 bmdickey17 2194@http://forums.flowingdata.com/ I put together an Excel doc with a Graph that showed a 91% correlation between Gas Prices and Chevron's Stock Price from Dec 2009 until Present. Has anyone else done any analysis like this or have any similar numbers? I can post my Excel Doc on Google Docs for everyone to review. I would love your feedback. Thanks. I did have problems formatting the data on Google Docs, I will try to work out the issues, but let me know what you think. <a href="https://spreadsheets.google.com/ccc?key=0AlEhmjhIhqtZdFg2dFlrYjlmaGEweW5HeERaWVJubVE&hl=en" rel="nofollow">https://spreadsheets.google.com/ccc?key=0AlEhmjhIhqtZdFg2dFlrYjlmaGEweW5HeERaWVJubVE&hl=en</a> When does amnt of skew/prevalence of outliers make median preferable over mean? http://forums.flowingdata.com/topic/when-does-amnt-of-skewprevalence-of-outliers-make-median-preferable-over-mean#post-2190 Thu, 28 Apr 2011 01:55:17 +0000 mehta_april 2190@http://forums.flowingdata.com/ I know that "deviations in the data are devil", and when the distribution is highly skewed, it is better to consider median as average rather than mean, but how to decide these hard-limits. For example: * CASE 1: * Assume X = 10,20,30,40,50,60,70 * In this case, I think that it is better to use mean and that it will give very accurate results. * CASE 2: * Assume X = 10,20,30,40,50,60,70,7000 * In this case, I think that it is better to use median instead of using the mean. * CASE 3: * Assume X = 10,20,30,400,500,600,700 * In this case, I think it is better to use IQR (Inter Quartile Range) But I'm stuck with how to decide these hard-limits i.e. which to use in which condition, in general. I've found a tool working on subjected principle, which takes context-less sample-distribution as input and determines whether mean is close/moderate or against the null-hypothesis. Find References:- * <a href="http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/MeanTest.htm" rel="nofollow">http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/MeanTest.htm</a> (For Mean Test) * <a href="http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/MediansTest.htm" rel="nofollow">http://home.ubalt.edu/ntsbarsh/Business-stat/otherapplets/MediansTest.htm</a> (For Median Test) What I'm really looking is a good answer which states how to derive these conclusions. Any help or learning pointers will be highly appreciated. Voronoi http://forums.flowingdata.com/topic/voronoi#post-2177 Fri, 15 Apr 2011 17:41:41 +0000 frmous 2177@http://forums.flowingdata.com/ I'm designing a UI that lets the user determine the proportions of 4 items. (possibly more in the future) I was impressed with all the UI's that let me build voronoi cells from points. Unfortunately, my user needs it easy. Drag on a cell, cell inflates, others deflate. Fin! The problem is I can't figure out a way to give my code a ratio and have it build a polygon based on how much area it should take up. Any ideas? I could end up doing the rectangular tree map instead. That seems easier to code, too bad the voronoi look astounding. Anyone have any ideas? Collecting Data Beer (Survey!) http://forums.flowingdata.com/topic/collecting-data-beer-survey#post-2162 Thu, 07 Apr 2011 07:24:05 +0000 Anon E Moose 2162@http://forums.flowingdata.com/ Hello, I really love this website and thought everyone might lend a hand and help me collect some data! I am doing some marketing research for school and would like you to complete a quick survey online: <a href="http://bit.ly/frZe57" rel="nofollow">http://bit.ly/frZe57</a> There is an option at the end to sign up for any free beer tastings in (if you're in the Atlanta area, and you must be 21 or over and have a valid ID). If you would like to remain anonymous don't put your email address at the end. Thanks, your opinion would be much appreciated and you could get information about a future brew night or tasting in your area! Parsons Journal for Information Mapping Call for Papers and Projects http://forums.flowingdata.com/topic/parsons-journal-for-information-mapping-call-for-papers-and-projects-1#post-2160 Wed, 06 Apr 2011 13:18:48 +0000 Katie4PJIM 2160@http://forums.flowingdata.com/ PJIM – Launching our new, interactive platform – submit your abstract today Abstracts due: May 16th 2011 Full submissions due: June 20th 2011 Already in our third year, The Parsons Journal for Information Mapping (PJIM) has published ground-breaking works from a truly international set of contributors. In every article and issue our contributors present their research, explorations, and implementations of innovative information design. Our contributors form an elite group of experts from a wide range of professional disciplines and academic institutions. With this coming issue – published April 18th – we are launching our brand new, interactive publishing platform. The platform is an archive of all past and present issues, allows our contributors and readers to interact online, supports advanced searching and filtering, and provides an array of options to read and share every article we publish. Our new web portal is designed to support our contributors' needs to publicize their work, gain international recognition, and interact and share ideas with our broad subscriber base. We are currently accepting new abstract submissions for our upcoming issue and looking for works focused on information design, data/knowledge visualization, taxonomies, data analytics, informatics, information systems, and graphic user interface design. If you are looking to leverage PJIM's interactive publishing platform and publicize your ground-breaking work to an international audience please submit your abstract today. To submit your work, please visit <a href="http://piim.newschool.edu/journal/submissions" rel="nofollow">http://piim.newschool.edu/journal/submissions</a>. PJIM is a quarterly publication of The Parsons Institute for Information Mapping – a research, development, and profession services facility in New York, NY. For more information please visit us at <a href="http://piim.newschool.edu" rel="nofollow">http://piim.newschool.edu</a> Discuss: Possible Shutdown of Data.gov and 7 other US Gov. open data sites http://forums.flowingdata.com/topic/discuss-possible-shutdown-of-datagov-and-7-other-us-gov-open-data-sites#post-2154 Tue, 05 Apr 2011 12:14:13 +0000 monserrate 2154@http://forums.flowingdata.com/ I just read Nathan's post on Guardian Datablog and thought I might try to gather the intelligent discussions around the web and see what people here would like to add: Discussions and Reporting on Data.gov Shutdown on web: <ul> <li><a href="http://www.guardian.co.uk/news/datablog/2011/apr/05/data-gov-crisis-obama" rel="nofollow">Guardian Datablog</a></li> <li><a href="http://news.ycombinator.com/item?id=2393875" rel="nofollow">HackerNews Discussion</a></li> <li><a href="http://www.readwriteweb.com/archives/datagov_7_other_sites_to_shut_down_after_budgets_c.php" rel="nofollow">ReadWriteWeb Post</a></li> <li><a href="http://sunlightfoundation.com/savethedata/" rel="nofollow">Sunlight Foundations "Save the Data" Petition</a></li> <li><a href="http://gov20.govfresh.com/congress-weighs-deep-cuts-to-funding-for-federal-open-government-data-platforms/" rel="nofollow">Alex Howard's Reporting on the topic on GovFresh</a></li> </ul> Key Points: <ol> <li>Budget would potentially be reduced from 37 million to 2 million</li> <li>Affected sites are Data.gov, USASpending.gov, Apps.gov, [what are the others?]</li> <li>Congress would vote on this cut in the next 4 months [exact date?]</li> </ol> How do I break into statistics, machine learning (next steps in education) http://forums.flowingdata.com/topic/how-do-i-break-into-statistics-machine-learning-next-steps-in-education#post-1862 Tue, 24 Aug 2010 23:09:02 +0000 sobral 1862@http://forums.flowingdata.com/ Hi everyone, I recently finished my bachelor's degree in computer science at Berkeley. Although it may be a bit late, I am just now getting interested in learning more about statistics, machine learning, and more math. Unfortunately, I don't have much of a math background (only took up to Linear Algebra) and the required probability/discrete math course for CS. I have already started working full time in a field that won't push me to learn about this area (work is mainly systems stuff). However, I am considering pursuing a MS in CS down the line because I do miss the academic environment. Probably not in the next year though. In the meantime, I would like to expose myself and learn about statistics, machine learning, and the required math it takes to understand these fields. Also, I would like to learn more about information visualization. I have some questions about all of this... 1) What are some good sources to get myself exposed to statistics, ML, required Math? I understand that there are many different areas, but a general guideline would be helpful. 2) Will an MS in CS with a concentration in AI/Machine Learning be enough of a background to work in "data science" as a career? Or will a PhD be preferred? I have limited research experience, but I already miss the academic setting of university. 3) How much artistic talent does it take to perform well in information visualization. I wouldn't consider myself a very artsy person. Sorry if this seems like a dumb question. Any advice/help would be greatly appreciated. Thanks! Serving by Weight http://forums.flowingdata.com/topic/serving-by-weight#post-2119 Tue, 01 Mar 2011 10:11:02 +0000 imrancs 2119@http://forums.flowingdata.com/ For example I have this data: headings = { :heading1 => { :weight => 60, :show_count => 0} :heading2 => { :weight => 10, :show_count => 0} :heading3 => { :weight => 20, :show_count => 0} :heading4 => { :weight => 10, :show_count => 0} } total_views = 0 Now I want to serve each heading based on their weightages. For instance, for first 10 requests/iterations, heading1, heading3, heading2 and heading4 would be served 6, 2, 1, and 1 times respectively in order (by weight). For every iteration show_count of served heading will increment by one and total_views will also increment globally. Could you please suggest an algorithm or some ruby code to handle this? O'Reilly' Strata Conference http://forums.flowingdata.com/topic/oreilly-strata-conference#post-2043 Sun, 16 Jan 2011 06:05:05 +0000 SmeeZee 2043@http://forums.flowingdata.com/ Anyone know if there is an equivalent in the UK? I would dearly love to go, but getting out to Santa Clara is quite an expense on top of the conference cost. If there is an equivalent, I would have thought there would be sufficient interest over here (in the UK)? Any thoughts?