Sabbatical July Report: My Year as a Data Scientist

I started working as a data scientist this June for my sabbatical project. Here is an update.

The Skills needed to be a data scientist

I continue to use Python (Pandas, in particular) daily. I have also needed to become familiar with other specialty software such as the Customer Relations Management software Salesforce.

Last week, I had to write my first for loop. Pandas does its best to allow you to avoid having to explicitly use them, but it has its limits. It doesn’t really feel like I am coding; it feels like I am mainly just doing a bunch of pandas commands to look at the data in a different way. Pandas makes things pretty easy.

Speaking of making things easy: the other people on the data team are working to use software so that everyone in the company can have access to the data via a drag-and-drop interface. I haven’t seen this software, so I can’t say much about it.

The most challenging part remains the banking knowledge. I have spent the last two weeks updating my code to look at finer data. Think about try to measure student learning and trying to decide on the unit of measurement. You could measure how the college is doing, how the classes are doing, or how each individual student is doing. I essentially made the switch to look at the analogue of “individual student” instead of “class.” This has both advantages and disadvantages, and we thought that it was worth the transition.

One thing that I continue to struggle with is knowing where data lives. There are fields in the database that are always empty, so I need to learn not to look there. There are also many similarly named fields, and I need to talk to people to learn about what they mean. In some cases, it seems like no one is really sure.

How academia and business are different.

I don’t have much to report on this this month. Because I have an unusual, one-year position, I think that my experience isn’t that different from doing research. I need to mostly create my own structure, I need to talk to people every couple of weeks to learn something, but I am largely working on my own. I am enjoying it.

I know that my experience isn’t typical for a data scientist, though. However, I can’t report on what I don’t know.

How will this experience influence my teaching?

One interesting thing that I have learned is that my urge to procrastinate is much higher than it has been in the last couple of years. I am able to control this through time blocking, but it is definitely a noticeable feeling. My best guess is that I am a lot less competent in this role, since I am new to data science. I think that my lack of expertise makes me want to avoid the work.

This gives me new empathy for my students, and it makes me realize focus is affected by competence. My students are new to the material in my classes (why else would they take the class?), so they likely have to fight procrastination at least as hard as I do now.

My feelings about being in industry.

See what I wrote for “How academia and business are different.”

Tags: Sabbatical