Transforming JSON data structure in Rails

I recently completed a project with a teammate at Flatiron where we developed a single page application using JavaScript and Rails. The requirements were pretty open-ended and we settled on building a page that allows the user to interact with multiple APIs from NASA Open API. First of all, I am really proud of how this turned out, please check it out on GitHub. My partner, Christopher Lempke, and I had a lot of fun and really stretched our legs with JavaScript and CSS (fetch galore, conditional rendering, drag and drop, animations, the jankiest guided walkthrough you have ever seen). Second, we got the opportunity to encounter what is a very common problem, working with similar data in very different formats.

Our page pulled information from APOD (astronomy picture of the day), NASA Image and Video Library, and Insight (mars weather). Well, Insight is currently not functioning, so for the mars weather we had to seed data from here (yay, backend API fetch!). Also, we gave users the ability to save images and display them on the screen. This last one was where our data manipulation issues came from. While the original source for all of the images was the NASA Image and Video Libary and we captured all of the information for each saved image in our Rails database, the structures were very different. Our database provided responses to our fetch requests that looked like this

A simple list of key, value pairs, easy. NASA on the other hand… organized the data a little more:

Now, the first thing I should have done when seeing this was to write a JavaScript function to map it to an easier format. But this provided me with an opportunity to practice working with nested data structures so we took what we were given and ran with it (lots of this.data[0].nasa_id and event.target.links[0].href).

But the problem came in when I wanted to reuse some of my functions written for the NASA images on images from our Rails database. Again, I should have (and would have if I had to actually deploy and maintain this project) gone back and transformed the NASA data, but instead, I wanted to see if I could update the Rails API response to mirror what NASA was providing. After much furious googling I determined that this was either, not typically done, or so simple that no one bothered to explain it. There were lots of questions about including data from linked tables,

render json: user.as_json(
       include:{ images: { except:  ["created_at", "updated_at"] }},
       except:  ["created_at", "updated_at"])

but nothing about what to do if images were not in the right format, so I put down google and actually thought about what I was trying to do. I had a Ruby hash (or an array of hashes) that I needed to manipulate. Once I actually looked at what I was doing it was straightforward, write a simple ruby method in my controller to manipulate the data,

and if it was an array of images, I used the same structure but just “mapped” the array.

So, what did I learn from this, first it would have been easier for me to simply transform the NASA data on the front end. That way all of my methods use a simple structure that is easy to read and update (also, if NASA updates their API, chances are I only have to update one function instead of 20). This also would have offloaded processing from my server to the user’s local computer, making the app more scalable. But since I am not deploying this, and instead this is a learning exercise the more important lesson what about how to approach and solve the problem. It helped me exercise my fundamentals (in this case Ruby data structures and manipulation) and reminded myself that not every answer is on StackOverflow. Next time I run into a problem, I will remember to first describe it in more basic terms, apply fundamentals, then if I don’t have a good answer move to StackOverflow.