-
Notifications
You must be signed in to change notification settings - Fork 0
BDMS-285: Well Inventory CSV validation/upload #145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
10d7ce8
feat: add formdata to custom method in ocotillo data provider
chasetmartin 7bb6615
feat: add well-inventory-bulk-import to AMPEditor list
chasetmartin 7d39f77
feat: add well-inventory-bulk-import to apps resource
chasetmartin a191eae
feat: add well-inventory-bulk-import route
chasetmartin fb8cf13
feat: MVP well inventory bulk import page
chasetmartin c5c75bb
fix: change button wording
chasetmartin d314721
feat: attempt at validation error handling
chasetmartin be211f5
feat: first attempt well inventory table entry
chasetmartin b1375f7
feat: move actions and csv message
chasetmartin 356a811
feat: install papaparse and npm audit fix
chasetmartin 33fd302
feat: move csv parse to utils folder
chasetmartin 48ac8e6
feat: use papaparse and move data grid defs
chasetmartin 45d18fd
feat: remove add row button to keep as validation engine for csv import
chasetmartin 1cb5945
feat: change instructions and no row display message
chasetmartin e785277
feat: add csv template button
chasetmartin 5b9069c
feat: add a no validation error found success chip
chasetmartin 95596a5
fix: change the no validation error message in chip
chasetmartin e8ef722
fix: move well inventory bulk import to top of apps
chasetmartin 89189cd
fix: change import to upload to avoid confusion in validation workflow
chasetmartin b277c32
feat: update schemas to better match pydantic api schemas
chasetmartin d322c7d
feat: move submit button above table
chasetmartin 9e064ba
refactor: move error and validation row mapping into utils
chasetmartin 7a27a0e
fix: removes field name parsing helper to keep the field name consist…
chasetmartin e7c3529
fix: adds return statement and notification if user is able to select…
chasetmartin ecc11cb
fix: removes leftover TODO note
chasetmartin 1e7f0c0
fix: move zod helpers into utils/zod folder
chasetmartin c2e40d1
fix: add returned type to handle csv import
chasetmartin 0c4c375
Merge branch 'staging' into cm-well-inventory-table-validation
chasetmartin 3dedcef
feat: add simple cypress test for happy path well inventory csv upload
chasetmartin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| /// <reference types="cypress" /> | ||
|
|
||
| describe('Well Inventory Bulk Import Page', () => { | ||
| beforeEach(() => { | ||
| cy.login() | ||
| cy.visit('/ocotillo/well-inventory-bulk-import') | ||
| }) | ||
|
|
||
| it('should render the well inventory bulk import page UI without errors', () => { | ||
| cy.get('input').should('have.length.at.least', 1) | ||
| cy.contains('Well Inventory Bulk Import').should('exist') | ||
| cy.get('label').contains(/upload csv/i).should('exist') | ||
| cy.get('button').contains(/submit/i).should('exist') | ||
| }) | ||
|
|
||
| it('should preserve CSV data through parse and unparse cycle', () => { | ||
| cy.readFile('cypress/fixtures/well-inventory.csv').then((originalCsv) => { | ||
| cy.intercept('POST', '**/well-inventory-csv', (req) => { | ||
| // For data sent as multipart string, check body contains CSV content | ||
| const body = req.body as string | ||
| const originalLines = originalCsv.split('\n') | ||
|
|
||
| // Check that key data from original CSV is in the submitted body | ||
| originalLines.forEach(line => { | ||
| if (line.trim()) { | ||
| //check all original csv fields for exact match in body | ||
| const fields = line.split(',') | ||
| fields.forEach(field => { | ||
| console.log("checking field: ", field.trim()) | ||
| expect(body).to.include(field.trim()) | ||
| }) | ||
| } | ||
| }) | ||
|
|
||
| req.reply({ | ||
| statusCode: 200, | ||
| body: { | ||
| summary: { total_rows_processed: 1, total_rows_imported: 1, validation_errors_or_warnings: 0 }, | ||
| wells: [], | ||
| validation_errors: [] | ||
| } | ||
| }) | ||
| }).as('submitCSV') | ||
|
|
||
| cy.get('input[type="file"]').selectFile('cypress/fixtures/well-inventory.csv', { force: true }) | ||
| cy.wait(1500) | ||
| cy.get('button').contains(/submit/i).click() | ||
| cy.wait('@submitCSV') | ||
| }) | ||
| }) | ||
| }) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| project,well_name_point_id,site_name,date_time,field_staff,utm_easting,utm_northing,utm_zone,elevation_ft,elevation_method,measuring_point_height_ft,measuring_point_description | ||
| Test Project_2,WELL-002,SITE-001,2024-01-15 10:30:00,John Smith,352342,4040485,13N,5500.5,Survey-grade GPS,2.5,Top of casing |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
57 changes: 57 additions & 0 deletions
57
src/pages/ocotillo/well-inventory-bulk-import/grid-defs.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import { GridColDef } from '@mui/x-data-grid' | ||
| import { IconButton } from '@mui/material' | ||
| import DeleteIcon from '@mui/icons-material/Delete' | ||
| import { allFieldNames, requiredFields, numericFields, booleanFields } from './utils' | ||
| import type { TableRow } from './index' | ||
|
|
||
| export function createGridColumns( | ||
| getCellError: (rowId: number, fieldName: string) => boolean, | ||
| handleDeleteRow: (id: number) => void | ||
| ): GridColDef<TableRow>[] { | ||
| // Create columns for all fields from schema | ||
| const dataColumns: GridColDef<TableRow>[] = allFieldNames.map((fieldName) => { | ||
| const isRequired = requiredFields.includes(fieldName) | ||
| const isNumeric = numericFields.includes(fieldName) | ||
| const isBoolean = booleanFields.includes(fieldName) | ||
|
|
||
| const baseColumn: GridColDef<TableRow> = { | ||
| field: fieldName, | ||
| headerName: fieldName, | ||
| width: 150, | ||
| editable: true, | ||
| cellClassName: (params) => { | ||
| return getCellError(params.row.id, fieldName) ? 'error-cell' : '' | ||
| }, | ||
| } | ||
|
|
||
| if (isNumeric) { | ||
| baseColumn.type = 'number' | ||
| baseColumn.width = 130 | ||
| } else if (isBoolean) { | ||
| baseColumn.type = 'boolean' | ||
| baseColumn.width = 120 | ||
| } | ||
|
|
||
| return baseColumn | ||
| }) | ||
|
|
||
| // Add actions column at the beginning | ||
| const actionsColumn: GridColDef<TableRow> = { | ||
| field: 'actions', | ||
| headerName: 'Actions', | ||
| width: 100, | ||
| sortable: false, | ||
| renderCell: (params) => ( | ||
| <IconButton | ||
| size="small" | ||
| onClick={() => handleDeleteRow(params.row.id)} | ||
| color="error" | ||
| > | ||
| <DeleteIcon /> | ||
| </IconButton> | ||
| ), | ||
| } | ||
|
|
||
| return [actionsColumn, ...dataColumns] | ||
| } | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The upload flow posts a
FormData(see handleSubmit creating a File and appending it to FormData), but the new Cypress test assumesreq.bodyis a CSV string and immediately casts it tostring. For multipart requestsreq.bodyis a FormData/Blob, so theincludeassertions will run against"[object FormData]"and always fail even when the CSV is sent correctly. This makes the added test a permanent false negative. Consider extracting the blob fromreq.body(e.g.,req.body.get('file')and reading it) before asserting on its contents.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I originally tried this approach, but the Cypress interceptor does not allow the .get('file'), the submission is just sent as a large string in the body in the Cypress environment, from what I can tell.