When I told friends I was starting a new job automating data processing, they wanted to know what language I’d be using. When I told them I’d be working with Visual Basic for Excel, I received more than one funny look. When I told them what kinds of problems I’d be solving, several people told me that Access or an SQL database was designed for doing those things.
Yes, this is absolutely true. I’ve used databases extensively for just these sorts of things. However, that’s missing the point entirely.
Not everyone who has to work with data is a data person.
Particularly in this age of data, the simple availability of data drives the creation of jobs, but not all of those jobs require people to be able to work with or even understand the underlying structures of the data they work with.
There are whole industries, for example, that boil down to helping people gain access to, understand, and sometimes change the data associated with them. That requires people skills, attention to detail, and knowledge of what our stored data means in the real world. It doesn’t require understanding relational databases or even the definition of a query. Even though the data in these industries is stored in relational databases and viewed and manipulated through queries, the users of this data don’t require those skills.
If you’re in one of these industries, you’ve built systems that do all that work and present the end-user with a graphical interface. But no system actually does all that work. At some point, decisions are made that this standard report in that format comes close enough to meeting everyone’s needs or that this bit of history should be stored but isn’t used often enough to be displayed in real time. At some point, you stop building out your system, and that point–because every change carries the possibility of breaking something, and because every change is something that requires testing when you make another change–is usually well before you’re done meeting everyone’s needs.
This is particularly true for meeting the needs of your problem-solvers and auditors. These are the people who work in the interface between your system and the people with whom you exchange data. These are the people who put human eyes on your data because they’re still better at detecting novel oddities and mistakes than any commercial computer. These are the people who figure out what’s missing when you receive only part of the data you need. These are the people who advocate for data feeds to be cleaned so corrections don’t revert to the error with every new upload.
These people are critical to smooth data operations, but they still don’t have to understand data structures. It’s much more important that they understand how people function: what kinds of mistakes they make, what pressure need to be applied to get a change made, how to develop the relationships with third-party data sources and recipients that ensure your company’s needs are treated as important. These are often people promoted from customer-facing or data entry positions because they demonstrated the skills required for the position. They were conscientious advocates of good data who spotted problems and got them fixed.
They also spent all their time working with that graphical interface. When faced with a program as flexible as Excel, some proportion of them will feel intimidated, even as they navigate the program just fine. Forget Access. Even if it weren’t an added expense to install it on the computer of a user whose primary function is people-oriented, getting these users comfortable enough to run an existing query, much less import and export data, is a significant hurdle. Access adds a layer of abstraction that isn’t necessarily appropriate for users whose relationship to data is much more concrete.
That shouldn’t prevent those users from having access to automation for the parts of their work that are rules-based, however. There are still significant savings to be had, in time, morale, and user errors, from making repetitive tasks disappear at the push of a button or from building functions that incorporate complex rules so they don’t have to be thought through every time.
What isn’t effectively done at the system level can be done outside of it with no extra load to that system. What won’t be done in a database program for fear of screwing something up will be done in a spreadsheet where the user can easily compare starting and end states until they become comfortable that they understand what a macro is doing.
To their credit, everyone I’ve explained this to has understood my point, but I think it’s worth saying more broadly. Automation is not about using the most powerful or specialized tool for the task. It isn’t (only or always) about creating tools for people with the same technological background as the automator. It is about giving users tools that fit into their workflow and require less work to use than they were already exerting. It’s about meeting users where they work.