Skip to content

Sql: Retrieve Foreign Key information from the database#346

Merged
Aniruddh25 merged 27 commits into
mainfrom
dev/anmunde/foreignKey
Apr 20, 2022
Merged

Sql: Retrieve Foreign Key information from the database#346
Aniruddh25 merged 27 commits into
mainfrom
dev/anmunde/foreignKey

Conversation

@Aniruddh25

Copy link
Copy Markdown
Collaborator

What is the change?

At bootstrap, in addition to retrieving table's primary keys and column information, this change adds functionality to retrieve foreign key information

  • all the foreign key names in the tables to be exposed
  • for each foreign key,
    -- the table it references,
    -- the referenced columns and
    -- the columns from the referencing table.

Why is this change needed?

We need to know the foreign key information since this is useful to write the correct queries to perform joins in GraphQL.

How is this implemented?

  • Added a new function SqlMetadataProvider::PopulateForeignKeyDefinitionAsync - takes in all the tables for which foreign key information is to be populated, invokes the methods to retrieve the information and fills in the ForeignKeyDefinition for each table. SqlGraphQLFileMetadataProvider calls this function at bootstrap
  • For PostgreSql, MsSql the query to get foreign key information is same and requires a 3-fold join, added that to the BaseSqlQueryBuilder.
    For MySql, the foreign key query only needs the view KEY_COLUMN_USAGE - added it to MySqlQueryBuilder
  • To filter for schema, we need to use database name as the value for TABLE_SCHEMA in MySql
  • For faster bootstrap, we issue a single query to the database and get information for all tables in one shot.

How is this tested?

  • Removed the foreign key information from sql-config.json and made sure existing tests don't regress.
  • Added a new assert to validate we retrieve the expected foreign key information in the SqlMetadataProviderTest. To know what we expect, made sure sql-config-test.json is updated.
  • Added a case where foreign key contains multiple referenced/referencing columns - see stocks_price_stocks_fk
  • Added a case where foreign key references a non-primary key unique column - see stocks_comics_fk

Remarks

  • After this change, the only additional information in the sql-config.json as compared to cosmos-config.json is the HttpVerbs section which can also be removed in favor of relying on sql-config-test.json for testing until the developer config change is checked in.
  • We can now generate a GQL schema from all the metadata information we have retrieved from database.

Comment thread DataGateway.Service/Resolvers/BaseSqlQueryBuilder.cs
@Aniruddh25 Aniruddh25 linked an issue Apr 16, 2022 that may be closed by this pull request
Comment thread DataGateway.Service/Resolvers/BaseSqlQueryBuilder.cs
Comment thread DataGateway.Service/Resolvers/BaseSqlQueryBuilder.cs Outdated
Comment thread DataGateway.Service/Resolvers/BaseSqlQueryBuilder.cs Outdated

@gledis69 gledis69 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition! Had just a couple of question/suggestions.

Comment thread DataGateway.Service/Resolvers/BaseSqlQueryBuilder.cs Outdated
Comment thread DataGateway.Service/Resolvers/MySqlQueryBuilder.cs Outdated
Comment thread DataGateway.Service/Services/MetadataProviders/SqlMetadataProvider.cs Outdated
@Aniruddh25 Aniruddh25 requested a review from aaronpowell April 20, 2022 03:50
@Aniruddh25 Aniruddh25 merged commit 76d0a54 into main Apr 20, 2022
@Aniruddh25 Aniruddh25 deleted the dev/anmunde/foreignKey branch April 20, 2022 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sql: Retrieve Foreign Key information from database

4 participants