Skip to content

BDMS-285: Well Inventory CSV validation/upload#145

Closed
chasetmartin wants to merge 29 commits into
stagingfrom
cm-well-inventory-table-validation
Closed

BDMS-285: Well Inventory CSV validation/upload#145
chasetmartin wants to merge 29 commits into
stagingfrom
cm-well-inventory-table-validation

Conversation

@chasetmartin

@chasetmartin chasetmartin commented Nov 26, 2025

Copy link
Copy Markdown
Collaborator

Why

This PR addresses the following problem / context:

How

Implementation summary - the following was changed / added / removed:

  • Uses PapaParse to parse the csv file and display the parsed data in a MUI DataGrid for display
  • Uses Zod schema to do instant validation of required fields and map the validation errors to rows in the DataGrid
  • DataGrid is editable
  • On submit turns the DataGrid back into a csv to send to the backend bulk csv endpoint
  • API validation errors are mapped back to the row/cell if 422 validation errors are returned so the user can edit and try again.
  • Adds simple Cypress test for well-inventory-bulk to check that csv uploaded contains all fields as csv submitted to backend on submit.

Notes

Any special considerations, workarounds, or follow-up work to note?

csv_upload.mp4

@chasetmartin chasetmartin requested review from TylerAdamMartinez and jirhiker and removed request for TylerAdamMartinez November 26, 2025 18:52

// Helper to convert field name to display name
// Converts snake_case to Title Case (e.g., "well_name_point_id" -> "Well Name Point Id")
const getDisplayName = (fieldName: string): string => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helper looks good, but it might make sense to move it into its own small util file (e.g., snakeToTitleCase.ts) so it can be reused wherever needed.

You may also want to add a guard to handle fieldName or name being null or undefined, or update the type signature to NonNullable<string> so the function guarantees it always receives a valid string. Adding a fallback return value or throwing a clear error will make this more robust and help avoid silent failures. Below is a code suggestion that includes a safety guard and a helpful function name change.

Suggested change
const getDisplayName = (fieldName: string): string => {
export const snakeToTitleCase = (name: string | null | undefined): string => {
if (!name) return '';
return name
.trim()
.split('_')
.filter(Boolean)
.map(w => w.charAt(0).toUpperCase() + w.slice(1))
.join(' ');
};

Finally, the transformation logic is straightforward, but you could make it slightly safer and more concise by trimming input and ignoring extra underscores (e.g., .filter(Boolean), which removes an additional space if the field name has two or more underscores), or by handling unexpected characters.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think I will just remove this helper. It seems best to keep the field name on the table matching the field name on the csv so there is no confusion. And it reduces the need for text manipulation.

@TylerAdamMartinez TylerAdamMartinez left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image

Looks good to me. When I submitted the "good" file, I got this error. Seems like the backend might have changed its requirement, but it proves that you're getting the backend errors and properly letting the user know. Had a few nitpicking comments and questions, but overall, Great work!

Comment thread src/pages/ocotillo/well-inventory-bulk-import/index.tsx
Comment thread src/pages/ocotillo/well-inventory-bulk-import/index.tsx
</Button>
</label>
<Link
href="https://docs.google.com/spreadsheets/d/1USYvb-A_-jsdY3qoPQBdkfjv5jjjYGaympp8SMAqJDo/edit?usp=sharing"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that Google will change this link in the future, rendering it dead?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a chance it would change, but this seems like a straightforward way for now. But open to other ideas.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After digging deeper into this, it seems that Google keeps their links quite stable. The FILE_ID is unlikely to change even if you modify the file's name, location, or permissions. If you want to prevent users from accidentally editing the file, you could update the link to a read-only format, like this: https://docs.google.com/spreadsheets/d/FILE_ID/view. Other than that, this approach looks good.

Comment thread src/pages/ocotillo/well-inventory-bulk-import/index.tsx
Comment thread src/pages/ocotillo/well-inventory-bulk-import/schema.ts
Comment thread src/providers/ocotillo-data-provider.ts Outdated
Comment thread src/pages/ocotillo/well-inventory-bulk-import/index.tsx Outdated
@TylerAdamMartinez

Copy link
Copy Markdown
Contributor
Captura de pantalla 2025-12-02 a la(s) 10 28 07

Once you fix the backend error in the review, it does submit without issue.

@TylerAdamMartinez

Copy link
Copy Markdown
Contributor

thing_202512021032.csv

This is more proof that it successfully created a new row in my local DB while testing this PR.

@chasetmartin chasetmartin marked this pull request as ready for review December 3, 2025 01:32
@chasetmartin

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +18 to +22
cy.intercept('POST', '**/well-inventory-csv', (req) => {
// For data sent as multipart string, check body contains CSV content
const body = req.body as string
const originalLines = originalCsv.split('\n')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Cypress intercept misreads multipart payload

The upload flow posts a FormData (see handleSubmit creating a File and appending it to FormData), but the new Cypress test assumes req.body is a CSV string and immediately casts it to string. For multipart requests req.body is a FormData/Blob, so the include assertions will run against "[object FormData]" and always fail even when the CSV is sent correctly. This makes the added test a permanent false negative. Consider extracting the blob from req.body (e.g., req.body.get('file') and reading it) before asserting on its contents.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally tried this approach, but the Cypress interceptor does not allow the .get('file'), the submission is just sent as a large string in the body in the Cypress environment, from what I can tell.

@TylerAdamMartinez TylerAdamMartinez left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall, looks good to me

@chasetmartin

Copy link
Copy Markdown
Collaborator Author

Closing this PR after the product discussion on Wednesday: getting data in > having a front end for getting data in. Keep this branch but focus work on CLI DataIntegrationGroup/OcotilloAPI#284

@chasetmartin chasetmartin deleted the cm-well-inventory-table-validation branch June 17, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants