Sabbatical July Report: My Year as a Data Scientist

I started working as a data scientist this June for my sabbatical project. Here is an update.

The Skills needed to be a data scientist

I continue to use Python (Pandas, in particular) daily. I have also needed to become familiar with other specialty software such as the Customer Relations Management software Salesforce.

Last week, I had to write my first for loop. Pandas does its best to allow you to avoid having to explicitly use them, but it has its limits. It doesn’t really feel like I am coding; it feels like I am mainly just doing a bunch of pandas commands to look at the data in a different way. Pandas makes things pretty easy.

Speaking of making things easy: the other people on the data team are working to use software so that everyone in the company can have access to the data via a drag-and-drop interface. I haven’t seen this software, so I can’t say much about it.

The most challenging part remains the banking knowledge. I have spent the last two weeks updating my code to look at finer data. Think about try to measure student learning and trying to decide on the unit of measurement. You could measure how the college is doing, how the classes are doing, or how each individual student is doing. I essentially made the switch to look at the analogue of “individual student” instead of “class.” This has both advantages and disadvantages, and we thought that it was worth the transition.

One thing that I continue to struggle with is knowing where data lives. There are fields in the database that are always empty, so I need to learn not to look there. There are also many similarly named fields, and I need to talk to people to learn about what they mean. In some cases, it seems like no one is really sure.

How academia and business are different.

I don’t have much to report on this this month. Because I have an unusual, one-year position, I think that my experience isn’t that different from doing research. I need to mostly create my own structure, I need to talk to people every couple of weeks to learn something, but I am largely working on my own. I am enjoying it.

I know that my experience isn’t typical for a data scientist, though. However, I can’t report on what I don’t know.

How will this experience influence my teaching?

One interesting thing that I have learned is that my urge to procrastinate is much higher than it has been in the last couple of years. I am able to control this through time blocking, but it is definitely a noticeable feeling. My best guess is that I am a lot less competent in this role, since I am new to data science. I think that my lack of expertise makes me want to avoid the work.

This gives me new empathy for my students, and it makes me realize focus is affected by competence. My students are new to the material in my classes (why else would they take the class?), so they likely have to fight procrastination at least as hard as I do now.

My feelings about being in industry.

See what I wrote for “How academia and business are different.”

Tags:

10 Responses to “Sabbatical July Report: My Year as a Data Scientist”

  1. Sabbatical August Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] « Sabbatical July Report: My Year as a Data Scientist […]

  2. Sabbatical September Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  3. Sabbatical October Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  4. Sabbatical November Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  5. Sabbatical December Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  6. January Sabbatical Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  7. February Sabbatical Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  8. March Sabbatical Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  9. April Sabbatical Report: My Year as a Data Scientist | Solvable by Radicals Says:

    […] July Report […]

  10. Solvable by Radicals Says:

    […] July Report […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: