ref(auto_source_config): Multiple calls does not create new repository #87437

armenzg · 2025-03-19T20:06:35Z

This depends on #87498 merging first.

In #86597 I introduced a bug causing many IntegrityErrors because we called create when the repository already existed.

This change is to have a test to prevent trying to create the repository more than once.

codecov · 2025-03-19T20:30:29Z

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #87437   +/-   ##
=======================================
  Coverage   87.73%   87.74%           
=======================================
  Files        9892     9897    +5     
  Lines      561283   561253   -30     
  Branches    22125    22112   -13     
=======================================
+ Hits       492465   492479   +14     
+ Misses      68416    68370   -46     
- Partials      402      404    +2

In #86597 I introduced a bug causing many [IntegrityErrors](https://sentry.sentry.io/issues/?environment=prod&groupStatsPeriod=auto&page=0&project=1&query=error.type%3AIntegrityError%20transaction%3Asentry.tasks.auto_source_code_config&referrer=issue-list&statsPeriod=90d) because we called `create` when the code mapping already existed. This change is to have a test to prevent trying to create the repository or code mappings more than once.

armenzg · 2025-03-20T17:28:30Z

src/sentry/issues/auto_source_code_config/task.py

            if not repository:
+                created = False


I'm refactoring some of the logic here to make sure the metrics are called correctly.

armenzg · 2025-03-20T17:29:01Z

src/sentry/issues/auto_source_code_config/task.py

-                project=project, stack_root=code_mapping.stacktrace_root
-            )
-            if existing_code_mappings.exists():
-                logger.warning("Investigate.", extra=extra)


Pass by change.

This has not been happening in production, thus, removing it.

armenzg · 2025-03-20T17:29:30Z

src/sentry/issues/auto_source_code_config/task.py

-                    continue
-
-                RepositoryProjectPathConfig.objects.create(
+                _, created = RepositoryProjectPathConfig.objects.get_or_create(


Changing from create to get_or_create.

armenzg · 2025-03-20T17:29:42Z

src/sentry/issues/auto_source_code_config/task.py

                if not dry_run:
-                    repository = Repository.objects.create(
+                    repository, created = Repository.objects.get_or_create(


Notice I'm changing from create to get_or_create.

armenzg · 2025-03-20T17:30:16Z

tests/sentry/issues/auto_source_code_config/test_process_event.py

@@ -91,8 +91,24 @@ def _process_and_assert_code_mapping(
            ),
            patch("sentry.utils.metrics.incr") as mock_incr,
        ):
+            repositories_count = Repository.objects.all().count()
+            code_mappings_count = RepositoryProjectPathConfig.objects.all().count()


These two variables are used below to determining if new code mappings and repositories have been created.

tests/sentry/issues/auto_source_code_config/test_process_event.py

MichaelSun48

Just the one comment in your tests. Otherwise lgtm.

Just curious — were there any lasting consequences here? Or did we just get a ton of Integrity errors because we tried to create repositories that already existed?

armenzg · 2025-03-21T13:49:47Z

Just the one comment in your tests. Otherwise lgtm.

Just curious — were there any lasting consequences here? Or did we just get a ton of Integrity errors because we tried to create repositories that already existed?

We get the errors, but the DB was not changed.

sentry-io · 2025-03-21T14:25:54Z

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

‼️ IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_repositoryproject_project_id_stack_root_e376b891_uniq"\nDETAIL: Key (project_id, stack_root)=(1825323, /web/wp/wp-includes/) already exists.\n') sentry.tasks.auto_source_code_config View Issue
‼️ IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_repositoryproject_project_id_stack_root_e376b891_uniq"\nDETAIL: Key (project_id, stack_root)=(6173372, ../var/h/deploy/airtable/current/) already exists.\n') sentry.tasks.auto_source_code_config View Issue
‼️ IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_repositoryproject_project_id_stack_root_e376b891_uniq"\nDETAIL: Key (project_id, stack_root)=(4506556170764288, /go/src/github.com/squareup/go-square/) already exists.\n') sentry.tasks.auto_source_code_config View Issue
‼️ IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_repositoryproject_project_id_stack_root_e376b891_uniq"\nDETAIL: Key (project_id, stack_root)=(4505781175255040, /var/task/src/controllers/worker/webpack:/roberto/) already exists.\n') sentry.tasks.auto_source_code_config View Issue
‼️ IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_repositoryproject_project_id_stack_root_e376b891_uniq"\nDETAIL: Key (project_id, stack_root)=(5275517, chalicelib/) already exists.\n') sentry.tasks.auto_source_code_config View Issue

_{Did you find this useful? React with a 👍 or 👎}

This fixes a bunch of integrity errors introduced in #87437. From the [docs](https://docs.djangoproject.com/en/5.1/ref/models/querysets/#get-or-create): > Any keyword arguments passed to get_or_create() — except an optional one called defaults — will be used in a get() call. If an object is found, get_or_create() returns a tuple of that object and False.

armenzg self-assigned this Mar 19, 2025

vercel bot deployed to Preview March 19, 2025 20:10 View deployment

armenzg force-pushed the fix/process_repos_and_code_mappings/armenzg branch from 3fa19fe to 43daf9a Compare March 20, 2025 15:16

github-actions bot added the Scope: Backend label Mar 20, 2025

vercel bot deployed to Preview March 20, 2025 15:20 View deployment

armenzg added 4 commits March 20, 2025 11:21

Minor renaming

2ea2a2d

WIP

bc6976e

Fix test

17241af

armenzg force-pushed the fix/process_repos_and_code_mappings/armenzg branch from 43daf9a to 4e86339 Compare March 20, 2025 17:27

armenzg changed the base branch from master to tests/trees/auto_source_config/armenzg March 20, 2025 17:27

Move the logic below

2cded45

armenzg commented Mar 20, 2025

View reviewed changes

armenzg marked this pull request as ready for review March 20, 2025 17:32

armenzg requested a review from a team as a code owner March 20, 2025 17:32

armenzg requested a review from MichaelSun48 March 20, 2025 17:32

vercel bot deployed to Preview March 20, 2025 17:36 View deployment

Fix test

c9208ca

vercel bot deployed to Preview March 20, 2025 18:08 View deployment

armenzg changed the title ~~tests(auto_source_config): Multiple calls does not create new repository~~ ref(auto_source_config): Multiple calls does not create new repository Mar 20, 2025

MichaelSun48 reviewed Mar 20, 2025

View reviewed changes

tests/sentry/issues/auto_source_code_config/test_process_event.py Show resolved Hide resolved

MichaelSun48 approved these changes Mar 20, 2025

View reviewed changes

Base automatically changed from tests/trees/auto_source_config/armenzg to master March 21, 2025 11:55

Merge branch 'master' into fix/process_repos_and_code_mappings/armenzg

a5410e3

armenzg enabled auto-merge (squash) March 21, 2025 12:09

vercel bot deployed to Preview March 21, 2025 12:14 View deployment

armenzg merged commit 0b751a2 into master Mar 21, 2025
48 checks passed

armenzg deleted the fix/process_repos_and_code_mappings/armenzg branch March 21, 2025 12:38

armenzg mentioned this pull request Mar 21, 2025

fix(auto_source_config): Use get_or_create properly #87581

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ref(auto_source_config): Multiple calls does not create new repository #87437

ref(auto_source_config): Multiple calls does not create new repository #87437

armenzg commented Mar 19, 2025 •

edited

Loading

codecov bot commented Mar 19, 2025 •

edited

Loading

armenzg Mar 20, 2025

armenzg Mar 20, 2025

armenzg Mar 20, 2025

armenzg Mar 20, 2025

armenzg Mar 20, 2025

MichaelSun48 left a comment •

edited

Loading

armenzg commented Mar 21, 2025

sentry-io bot commented Mar 21, 2025 •

edited

Loading

ref(auto_source_config): Multiple calls does not create new repository #87437

ref(auto_source_config): Multiple calls does not create new repository #87437

Conversation

armenzg commented Mar 19, 2025 • edited Loading

codecov bot commented Mar 19, 2025 • edited Loading

Codecov Report

armenzg Mar 20, 2025

Choose a reason for hiding this comment

armenzg Mar 20, 2025

Choose a reason for hiding this comment

armenzg Mar 20, 2025

Choose a reason for hiding this comment

armenzg Mar 20, 2025

Choose a reason for hiding this comment

armenzg Mar 20, 2025

Choose a reason for hiding this comment

MichaelSun48 left a comment • edited Loading

Choose a reason for hiding this comment

armenzg commented Mar 21, 2025

sentry-io bot commented Mar 21, 2025 • edited Loading

Suspect Issues

armenzg commented Mar 19, 2025 •

edited

Loading

codecov bot commented Mar 19, 2025 •

edited

Loading

MichaelSun48 left a comment •

edited

Loading

sentry-io bot commented Mar 21, 2025 •

edited

Loading