Counting Down the Days »« The Reading List, 6/22/2014

Meeting Users Where They Work

When I told friends I was starting a new job automating data processing, they wanted to know what language I’d be using. When I told them I’d be working with Visual Basic for Excel, I received more than one funny look. When I told them what kinds of problems I’d be solving, several people told me that Access or an SQL database was designed for doing those things.

Yes, this is absolutely true. I’ve used databases extensively for just these sorts of things. However, that’s missing the point entirely.

Not everyone who has to work with data is a data person.

Particularly in this age of data, the simple availability of data drives the creation of jobs, but not all of those jobs require people to be able to work with or even understand the underlying structures of the data they work with.

There are whole industries, for example, that boil down to helping people gain access to, understand, and sometimes change the data associated with them. That requires people skills, attention to detail, and knowledge of what our stored data means in the real world. It doesn’t require understanding relational databases or even the definition of a query. Even though the data in these industries is stored in relational databases and viewed and manipulated through queries, the users of this data don’t require those skills.

If you’re in one of these industries, you’ve built systems that do all that work and present the end-user with a graphical interface. But no system actually does all that work. At some point, decisions are made that this standard report in that format comes close enough to meeting everyone’s needs or that this bit of history should be stored but isn’t used often enough to be displayed in real time. At some point, you stop building out your system, and that point–because every change carries the possibility of breaking something, and because every change is something that requires testing when you make another change–is usually well before you’re done meeting everyone’s needs.

This is particularly true for meeting the needs of your problem-solvers and auditors. These are the people who work in the interface between your system and the people with whom you exchange data. These are the people who put human eyes on your data because they’re still better at detecting novel oddities and mistakes than any commercial computer. These are the people who figure out what’s missing when you receive only part of the data you need. These are the people who advocate for data feeds to be cleaned so corrections don’t revert to the error with every new upload.

These people are critical to smooth data operations, but they still don’t have to understand data structures. It’s much more important that they understand how people function: what kinds of mistakes they make, what pressure need to be applied to get a change made, how to develop the relationships with third-party data sources and recipients that ensure your company’s needs are treated as important. These are often people promoted from customer-facing or data entry positions because they demonstrated the skills required for the position. They were conscientious advocates of good data who spotted problems and got them fixed.

They also spent all their time working with that graphical interface. When faced with a program as flexible as Excel, some proportion of them will feel intimidated, even as they navigate the program just fine. Forget Access. Even if it weren’t an added expense to install it on the computer of a user whose primary function is people-oriented, getting these users comfortable enough to run an existing query, much less import and export data, is a significant hurdle. Access adds a layer of abstraction that isn’t necessarily appropriate for users whose relationship to data is much more concrete.

That shouldn’t prevent those users from having access to automation for the parts of their work that are rules-based, however. There are still significant savings to be had, in time, morale, and user errors, from making repetitive tasks disappear at the push of a button or from building functions that incorporate complex rules so they don’t have to be thought through every time.

What isn’t effectively done at the system level can be done outside of it with no extra load to that system. What won’t be done in a database program for fear of screwing something up will be done in a spreadsheet where the user can easily compare starting and end states until they become comfortable that they understand what a macro is doing.

To their credit, everyone I’ve explained this to has understood my point, but I think it’s worth saying more broadly. Automation is not about using the most powerful or specialized tool for the task. It isn’t (only or always) about creating tools for people with the same technological background as the automator. It is about giving users tools that fit into their workflow and require less work to use than they were already exerting. It’s about meeting users where they work.

Comments

  1. medivh says

    I have to admit, I would have been one of the people looking at you funny for saying that you do DP automation in VBA. There’s a certain culturally taught knee-jerk reaction to VB (and it’s “for Applications” offshoot) among many who got into programming via geekery, or who try to assimilate into that culture later. But it really is about using the right tool for the job, cultural bias be damned.

    Also, my immediate instinct to solve the problem you’ve already solved would be to build a custom app that imports data from an excel file and communicates with some sort of SQL database. Also with a GUI interface that lets you type in the data directly. But then I have to remember to say to myself, “gloves.” I keep getting reminded, and keep falling into the same trap regardless…

  2. Ysanne says

    Automating stuff that Excel is actually good at is definitely worthwhile anyway. And even for the things Excel isn’t really meant for it’s definitely better than nothing. But…
    Having done some serious number crunching in Excel with VBA in the past few years because “that’s the only tool our clients know”, I understand your argument and yes, sometimes you can’t get around using Excel. Unfortunately that’s the sole argument going for it, because Excel is definitely not suited to dealing with the amounts of data we routinely have and neither is VBA. There is only so much you can trick your way around with playing to the strengths of the calculation engine, disabling automatic recalculation and using arrays instead of worksheets.
    Fortunately at my workplace the decision-makers have finally understood that it’s really only the user-facing frontend of our analysis tools that needs to be Excel, so we’ve finally started to use it as exactly that. For example, putting the actual data into a database and having Excel only get out (and put back in) the portions it actually needs to use makes everything SO much faster despite the added import/export steps. Likewise, the ability to connect to R and run stats in there is a life-saver.
    Good luck with whatever you need to code up!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>