Um ... Microsoft?

SQL, Python. And Excel to publish the results to outsiders.

Pandas profiling reports

Google search and Reddit search, because I’m absolutely sure we just had a thread about this recently

R

Depends on the amount of data and how complex the analysis and if it’s a one time thing or needs an automated output.

Needs any automation - probably Tableau. Try to give it to the BI team if it’s a straightforward ask.

Not very complex, no automation, can open the dataset in Excel - then probably Excel

No automation but big dataset or needs cleaning or feature engineering or lots of exploration - Python, mostly Pandas and Seaborn, in a Jupyter notebook.

I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:

• [/r/datascienceproject] What data science tools have the best user experience? (r/DataScience)

 ^(If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads.) ^(Info ^/ ^Contact)

Databricks, snowflake, vscode with the right extensions

tidyverse, duckdb, quarto. Haven't tried GitHub copilot, but it looks amazing.

Your project sounds like Metaflow to me, which is what we use. Seems kinda excessive that you guys are building it from the ground up when there are already good tools out there.

i've used coiled before - kinda frustrating to get used to

Excel, and selectively Pandas.

Excel with a sprinkling of R.

Excel.

AWS Batch and EMR jobs using docker containers

Distributing work over clusters to run in parallel. It's commonly used with Spark, idk about other frameworks but pretty sure there's a wide support.

Elastic Map Reduce. Allows you to distribute computing over a cluster of machines. We use it to run Spark mostly.

The ‘job’ refers to us using job queues. We build a job pipeline and then execute it when needed. AWS takes care of getting the compute resources and then shuts them down when the job is finished

For me, the biggest leap in DS was finding good tools to build data models, I love siren.io for this. It helps rapidly develop an understanding of the features and relations between different data structures. Once you have a great model, it’s possible to advance the level of analytics using a platform like cogility.io.

Libre Office / Open Office Calc ( Spreadsheet)

Hex all the way! Sql & Python combined in one notebook

Sql + any sql client would be my vote.

R or Python ... SQL to query your data for analysis. Are you all using SQL to analyze your data?!

Depending on the size of your data you may want to run your analysis via Databricks.

Sql + Excel

Hmm no aws tools

Very Azure-centric. Weird list.

Don't love this list.

Just 50? :p

A report documentor. Goes through the data calls out the routine key points and draws pretty chart so we can focus on the deep dives.

Something that’ll take SQL queries and convert it to DAX

A good data annotation software where multiple people can label and annotate data collaboratively

Some datasets are sensitive in nature and you wouldn’t want people outside of your company to view them though.

Google earth engine is amazing! Wish there were more platforms like it for other datasets too

I like it, but I can't use it in my company as I believe it doesn't with gitlab.

I like it, but I can't use it as I believe it is not available for gitlab.

Access to the access management platform that grants access to my access management platform.

Such a timely request. Here’s what I recently built. You can connect your notebook to a GitHub repo and the tool will automatically version every modeling iteration and show the code diff in GitHub.

https://www.reddit.com/r/learnmachinelearning/comments/qwiwpg/reimagined_ml_training/?utm_source=share&utm_medium=ios_app&utm_name=iossmf

This is great! I recommend looking on Ploomber(https://ploomber.io ), for a more robust integration between the two. It has native integration with Git and it does some of this work for you, the Jupyter plugin allows you to read as a notebook and it allows to actually track the notebooks within git without having a change in every commit (because of the notebook outputs and its metadata). It does the heavy lifting for you.

This is a thing for Observable notebooks. But the language is JavaScript specific I think.

TBH I haven't used a notebook in years just it's a commonly requested format for management when they want a presentation of data in a format they can easily understand.

You forgot the big one https://flourish.studio/

Links:

• Streamlit
• Diagrams.net
• Genially
• Doodly
• Visme
• Polymer Search

I use draw.io for flowcharts for the same reasons as you mention. But it’s good to know a back up!

Thanks for the suggestions bud

How do you deploy streamlit apps? Do you have to use their service?

Nice list. Have you also tried Flourish or Data Wrapper? What did you think?