Do not copy from examples. Example only covers steps 1 & 2. use your Python know

Posted: April 22nd, 2022

Do not copy from examples. Example only covers steps 1 & 2.
use your Python knowledge and skills to design, execute, and present the results of a Data ETL pipeline project. You will create original code and a written report summarizing the goals, methods, and results of your project.
The project begins with an exploration of the New York Times API endpoints
(Links to an external site.) https://developer.nytimes.com/
. You might have an idea about what kind of data you want to analyze or what kind of question you want to answer. But if the data you need is not available, then you are only setting yourself up for a very challenging – perhaps impossible – final project. Therefore, your first step is to identify an actionable goal, e.g., a question to answer, for your project by exploring the documentation for the New York Times API
(Links to an external site.) https://developer.nytimes.com/
.
With that question in mind, it’s time to sketch out each of the stages in your ETL pipeline. The main questions for which you need to provide answers are as follows:
Step 1: Describe your project (4 lines to a paragraph each)
1.What is the goal or question of your project?
2.Where is the data I need? Be specific! Which API endpoint? Are the important pieces of data I need to extract from the API actually available from the endpoint(s) I have identified?
For this question, please explain how to generate the API URL. I need to be able to recreate it. If possible please list the steps and text you used to create the API URL.
3. What do I need to do to transform the raw data extracted from the API into something usable? You do not need to be writing code yet! In fact, thinking in Python terms at this stage could be counter-productive. Just think in human terms for now: What about your data needs to be transformed? What parts do you need to keep and what can you discard?
ETL projects often require multiple transform stages. For example, one transform stage that cleans and prepares the extracted data, and another that uses built-in Python and/or third-party libraries to transform and even generate new data.
4. What will I load my data into, and how? Your raw data, once processed in your transform stages, is going to be loaded into a file or database depending on what software you plan to use to present your results. How should you store it? What format and structure should the data be stored in?
Step 2: Write and run your code example
Checklist for your code (Have to use JSON)
Required. You will not pass this assignment without meeting these requirements:
Your code executes successfully
Your code extracts data from an external source
Your code transforms data from its raw form to something more directly useful
Your code loads data into some format, file, or system that a human can use to interpret your output.
Using file I/O or database operations to store data
Use of one or more third-party module(s)
Functionality (doing things in your code that demonstrate the ability to use Python constructs like variables, flow control, etc.)
Code structure, including the use of functions
Code comments and style
Step 3: Create code-images and write your report
Summarize your project in a short written report. Here are the contents you need to produce:
A Summary of the purpose or goal of your project, the technologies used, and the results. This shouldn’t take more than a paragraph, but you should write as much as you feel is necessary to summarize your project.
A series of Code images using Carbon
(Links to an external site.) https://carbon.now.sh/
to display the important parts of your code. What are the important parts? That’s for you to decide – it should not be every line of code, but what lines do important things in the context of your project?
Include Code descriptions based on each code image.
Wrap up with more ideas about what you would do next if you had to (or chose to) do a follow-up project.

Expert paper writers are just a few clicks away

Place an order in 3 easy steps. Takes less than 5 mins.

Calculate the price of your order

You will get a personal manager and a discount.
We'll send you the first draft for approval by at
Total price:
$0.00