DIAL Solutions Catalog: create parser for ETL framework

About Me
This is Miranda Ren, a sophomore at Cornell University studying cs.
This is my first project with GoSC and thanks for the opportunity!

My Project
This project is to build a JSON-file parser for ETL framework to alleviate the need to write specific ingestion tasks each time, and have a generic ingestion framework instead that can more easily accommodate new data sources as we add them to the catalog.
These project goals were set based on discussion with mentor and original proposal:

  • Implement function/module to create project from JSON file
  • Create page in Rails app allowing a user to import product or project JSON file
  • Create example Rake tasks for importing csv or other files

My Timezone
GMT + 8

Getting in touch

Week 1- 3

  • Set up communications with mentor to discuss project details and create a website for sharing updating progress in future.

  • Installed and set environment for Ruby, HomeBrew, rbenv, Rails, and PostgreSQL, as well as setup SSH key on GitHub.

  • Got familiar with Ruby syntax and grammar by watching tutorials, read guides, and practice coding.

  • Wrote small Rake projects to practice on RoR.

Week 4 Updates (July 4)

  • created file in Ruby to parse JSON format code and practice extracting data from sample JSON file of project record

  • generated controller using Ruby on Rails in a Rail app for JSON parsing

  • working on exporting and creating records that can be shown in the web page (in html.erb format)

Plan for next week

  • Finish data exporting and record creating onto the rail app

  • Learn more about styling in Rail app