Skip to content

jir-transducer-data#184

Closed
jirhiker wants to merge 32 commits into
stagingfrom
jir-transducer-data
Closed

jir-transducer-data#184
jirhiker wants to merge 32 commits into
stagingfrom
jir-transducer-data

Conversation

@jirhiker

@jirhiker jirhiker commented Oct 10, 2025

Copy link
Copy Markdown
Member

Why

This PR addresses the following problem / context:

  • Need to transfer WaterLevelsContinous_Pressure to Ocotillo

How

Implementation summary - the following was changed / added / removed:

Notes

Any special considerations, workarounds, or follow-up work to note?

  • Use bullet points here

Comment thread services/util.py Dismissed
Comment thread services/util.py Dismissed
Comment thread services/util.py Dismissed
@jirhiker jirhiker changed the base branch from production to staging October 10, 2025 19:42
@codecov-commenter

codecov-commenter commented Oct 10, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.46758% with 25 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
core/initializers.py 77.14% 8 Missing ⚠️
services/observation_helper.py 76.00% 6 Missing ⚠️
core/permissions.py 63.63% 4 Missing ⚠️
schemas/__init__.py 85.00% 3 Missing ⚠️
core/app.py 50.00% 2 Missing ⚠️
db/transducer.py 92.30% 2 Missing ⚠️
Files with missing lines Coverage Δ
api/observation.py 100.00% <100.00%> (ø)
api/search.py 97.82% <100.00%> (ø)
api/thing.py 96.52% <100.00%> (ø)
core/dependencies.py 100.00% <100.00%> (ø)
core/enums.py 100.00% <100.00%> (ø)
db/__init__.py 97.67% <100.00%> (ø)
db/sensor.py 100.00% <100.00%> (ø)
db/thing.py 100.00% <100.00%> (ø)
main.py 81.81% <ø> (ø)
schemas/contact.py 100.00% <100.00%> (ø)
... and 20 more

@jacob-a-brown jacob-a-brown left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have a separate table for transducer observations?

@ksmuczynski

Copy link
Copy Markdown
Contributor

@jacob-a-brown Because it's storing time series data. If continuous data were captured and stored in the same manner as our discrete samples, every 15-minute reading from a pressure transducer would create a new FieldEvent, FieldActivity, Sample, and Observation record. The core tables would become massive and slow, not to mention redundant.

I think a separate table is def the way to go, but I have some minor comments that I'm working on writing up and integrating into the review :)

@jacob-a-brown

Copy link
Copy Markdown
Contributor

When we retrieve all observations from a well, should it also retrieve the continuous observations? or will that be separate?

@ksmuczynski

Copy link
Copy Markdown
Contributor

@jacob-a-brown I think it should return the continuous data, but aggregated based on the length of a user's requested date range. Like, if the date range is long (e.g., > 1 month), then the raw 15-minute data is not returned. The API could ask the database to calculate daily or weekly averages. If the range is short (e.g., < 3 days), the API can then decide it's safe to fetch the raw, continuous data.

@jirhiker

Copy link
Copy Markdown
Member Author

There likely is a business use case for correlated continuous and manual waterlevel measurements, and time series statistics.

Comment thread db/transducer.py
Comment thread db/__init__.py

@ksmuczynski ksmuczynski left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment/suggestion in the db/transduce.py file about referencing the Deployment table instead of the Thing table.

@jirhiker

Copy link
Copy Markdown
Member Author

@jacob-a-brown have a look how I added dynamic enum generation for use with Pydantic validation

@jacob-a-brown jacob-a-brown left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the use of enums to validate POST and PATCH data. This can (and should) be used elsewhere.

Comment thread core/enums.py
Comment thread db/transducer.py Outdated
Comment thread db/transducer.py Outdated
Comment thread db/transducer.py
Comment thread db/transducer.py
Comment thread db/transducer.py Outdated
Comment thread tests/test_contact.py Outdated
@jirhiker jirhiker marked this pull request as ready for review October 20, 2025 19:50
@jirhiker jirhiker requested a review from ksmuczynski October 20, 2025 19:50
Comment thread db/transducer.py Outdated
Comment thread db/transducer.py
Comment thread db/transducer.py Outdated
@jirhiker jirhiker requested a review from ksmuczynski October 21, 2025 06:01
@jirhiker

Copy link
Copy Markdown
Member Author

#197 Has these changes

@jirhiker jirhiker closed this Oct 24, 2025
@jirhiker jirhiker deleted the jir-transducer-data branch December 3, 2025 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants