In Part I, I presented a high-level view of my project’s architecture, and outlined the questions any data migrator should be asking themselves. I also shared some of the choices we made, along with the rationale for each decision. If you haven’t read the first part of this story yet, I would recommend checking that out before continuing.
For the last 6 months, my team has been implementing a solution for migrating our client’s Oracle database into Databricks delta. In that time we’ve focused mainly on the following:
There has been an explosion of cutting edge data solutions in the 2010s, many of which target the growing problem of scalable storage with big data. Technologies like these are built on the shoulders of giants, immensely complex behind the scenes, but powerful and manageable to use once set up. The tricky part is that set up: how can we efficiently and reliably migrate data to these new powerful solutions?
The goal of this article is to outline a success story on one of those migration techniques, hopefully filling in the gaps where documentation is a bit too pedantic. This…
When I was a junior in high school, my AP Lit teacher convinced me to enter a local writing contest sponsored by the Arlington Historical Society. That year I had transformed my shitty teenage writing skills into something that my mom, a former journalist turned reading teacher, could read without a sense of deep disappointment.
I published the original essay, unedited, to Medium here. The prompt was this:
Was Arlington County Heroic or Quixotic in its attempts to integrate schools in the 1950s?
Below is a piece I wrote for the Arlington Historical Society, in the Spring of 2014. I made no further edits, and besides adding pictures this is what was released in 2014 with the AHS annual publication.
Last Friday, at approximately 10:45, first period at Washington-Lee High School took a brief hiatus for the annual black heritage assembly. For half an hour, the auditorium was crowded with an audience of just under a thousand teenagers. They sat attentively, with the occasional whisper, as they listened to skits, poetry, and anecdotes about people ranging from Malcolm X, to Rosa Parks, to…
Chemical Engineer by education, Data Engineer by trade. Easily excited, mainly by topics from endurance sports, to pour-over coffee, to biotech trends