Added Golden Checks for EIA Imports [EIA_Electricity, EIA_Coal, EIA_Petroleum, EIA_NaturalGas, EIA_International, EIA_TotalEnergy, EIA_SEDS, EIA_NuclearOutages] and CensusCountyBusinessPatterns#2101
Conversation
Code fix unenergy
There was a problem hiding this comment.
Code Review
This pull request adds validation configurations and golden observation/summary files for several US EIA open data imports and New York education datasets. While many golden files were successfully added, several referenced golden summary report files are missing from the PR, specifically those for EIA Electricity, EIA Petroleum, EIA Total Energy, and New York English and Math education data. These missing files must be generated and included to ensure the validation checks can run successfully.
|
Few pointers to take into consideration:
|
Updated the PR with -
|
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request introduces validation configurations and golden data files for Census County Business Patterns and various US EIA OpenData datasets. However, several referenced golden files are missing from the pull request, specifically the golden summary reports and observations for Census County Business Patterns, as well as the golden summary reports for EIA Electricity, EIA Petroleum, and EIA Total Energy.
|
LGTM |
This PR introduces golden checks for EIA Imports and CensusCountyBusinessPatterns. By adding these golden test files, we ensure that future data imports and processing scripts produce consistent output, preventing unexpected regressions.
Key Changes
Added Golden Files: Generated and added golden test files for EIA imports, and CensusCountyBusinessPatterns data.
Test Results (validation_output.csv)-
EIA_Electricity - link SOP
EIA_Coal - link SOP
EIA_Petroleum - link SOP
EIA_NaturalGas- link SOP
EIA_SEDS - link SOP
EIA_NuclearOutages - link SOP
EIA_TotalEnergy - link SOP
EIA_International - link SOP
NOTE - For EIA_International, cron scheduler is corrected as it was previously not working. Also, a CL is raised to resolve the missing_references issue. link
CensusCountyBusinessPatterns - link SOP