Skip to content

Commit 9bf1d71

Browse files
authored
add async124 async-function-could-be-sync (#309)
* add async124 async-function-could-be-sync * fixed two bugs in async91x * enable no_checkpoint_warning_decorators for async124. Remove unused Visitor91x.safe_decorator. * mention ASYNC124 in docs/usage.rst, and convert mentions of PEP to :pep:
1 parent 1122f25 commit 9bf1d71

File tree

11 files changed

+361
-35
lines changed

11 files changed

+361
-35
lines changed

docs/changelog.rst

+7-1
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ Changelog
44

55
`CalVer, YY.month.patch <https://calver.org/>`_
66

7+
25.1.1
8+
=======
9+
- Add :ref:`ASYNC124 <async124>` async-function-could-be-sync
10+
- :ref:`ASYNC91x <ASYNC910>` now correctly handles ``await()`` in parameter lists.
11+
- Fixed a bug with :ref:`ASYNC91x <ASYNC910>` and nested empty functions.
12+
713
24.11.4
814
=======
915
- :ref:`ASYNC100 <async100>` once again ignores :func:`trio.open_nursery` and :func:`anyio.create_task_group`, unless we find a call to ``.start_soon()``.
@@ -238,7 +244,7 @@ Changelog
238244

239245
22.9.2
240246
======
241-
- Fix a crash on nontrivial decorator expressions (calls, PEP-614) and document behavior.
247+
- Fix a crash on nontrivial decorator expressions (calls, :pep:`614`) and document behavior.
242248

243249
22.9.1
244250
======

docs/rules.rst

+9-2
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ _`ASYNC100` : cancel-scope-no-checkpoint
1919
_`ASYNC101` : yield-in-cancel-scope
2020
``yield`` inside a :ref:`taskgroup_nursery` or :ref:`timeout_context` is only safe when implementing a context manager - otherwise, it breaks exception handling.
2121
See `this thread <https://discuss.python.org/t/preventing-yield-inside-certain-context-managers/1091/23>`_ for discussion of a future PEP.
22-
This has substantial overlap with :ref:`ASYNC119 <ASYNC119>`, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 <https://peps.python.org/pep-0533/>`_.
22+
This has substantial overlap with :ref:`ASYNC119 <ASYNC119>`, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by :pep:`533`.
2323

2424
_`ASYNC102` : await-in-finally-or-cancelled
2525
``await`` inside ``finally``, :ref:`cancelled-catching <cancelled>` ``except:``, or ``__aexit__`` must have shielded :ref:`cancel scope <cancel_scope>` with timeout.
@@ -76,7 +76,7 @@ ASYNC118 : cancelled-class-saved
7676

7777
_`ASYNC119` : yield-in-cm-in-async-gen
7878
``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed.
79-
We strongly encourage you to read `PEP 533 <https://peps.python.org/pep-0533/>`_ and use `async with aclosing(...) <https://docs.python.org/3/library/contextlib.html#contextlib.aclosing>`_, or better yet avoid async generators entirely (see `ASYNC900`_ ) in favor of context managers which return an iterable :ref:`channel/stream/queue <channel_stream_queue>`.
79+
We strongly encourage you to read :pep:`533` and use `async with aclosing(...) <https://docs.python.org/3/library/contextlib.html#contextlib.aclosing>`_, or better yet avoid async generators entirely (see `ASYNC900`_ ) in favor of context managers which return an iterable :ref:`channel/stream/queue <channel_stream_queue>`.
8080

8181
_`ASYNC120` : await-in-except
8282
Dangerous :ref:`checkpoint` inside an ``except`` block.
@@ -95,6 +95,13 @@ _`ASYNC123`: bad-exception-group-flattening
9595
Dropping this information makes diagnosing errors much more difficult.
9696
We recommend ``raise SomeNewError(...) from group`` if possible; or consider using `copy.copy` to shallow-copy the exception before re-raising (for copyable types), or re-raising the error from outside the `except` block.
9797

98+
_`ASYNC124`: async-function-could-be-sync
99+
Triggers when an async function contain none of ``await``, ``async for`` or ``async with``.
100+
Calling an async function is slower than calling regular functions, so if possible you
101+
might want to convert your function to be synchronous.
102+
This currently overlaps with :ref:`ASYNC910 <ASYNC910>` and :ref:`ASYNC911 <ASYNC911>` which, if enabled, will autofix the function to have :ref:`checkpoint`.
103+
This excludes class methods as they often have to be async for other reasons, if you really do want to check those you could manually run :ref:`ASYNC910 <ASYNC910>` and/or :ref:`ASYNC911 <ASYNC911>` and check the methods they trigger on.
104+
98105
Blocking sync calls in async functions
99106
======================================
100107

docs/usage.rst

+3-3
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ adding the following to your ``.pre-commit-config.yaml``:
3333
minimum_pre_commit_version: '2.9.0'
3434
repos:
3535
- repo: https://github.com/python-trio/flake8-async
36-
rev: 24.11.4
36+
rev: 25.1.1
3737
hooks:
3838
- id: flake8-async
3939
# args: [--enable=ASYNC, --disable=ASYNC9, --autofix=ASYNC]
@@ -215,11 +215,11 @@ Example
215215
``no-checkpoint-warning-decorators``
216216
------------------------------------
217217

218-
Comma-separated list of decorators to disable checkpointing checks for, turning off :ref:`ASYNC910 <async910>` and :ref:`ASYNC911 <async911>` warnings for functions decorated with any decorator matching against an entry in the list.
218+
Comma-separated list of decorators to disable checkpointing checks for, turning off :ref:`ASYNC910 <async910>`, :ref:`ASYNC911 <async911>`, and :ref:`ASYNC124 <async124>` warnings for functions decorated with any decorator matching against an entry in the list.
219219
Matching is done with `fnmatch <https://docs.python.org/3/library/fnmatch.html>`_.
220220
Defaults to disabling for ``asynccontextmanager``.
221221

222-
Decorators-to-match must be identifiers or dotted names only (not PEP-614 expressions), and will match against the name only - e.g. ``foo.bar`` matches ``foo.bar``, ``foo.bar()``, and ``foo.bar(args, here)``, etc.
222+
Decorators-to-match must be identifiers or dotted names only (not :pep:`614` expressions), and will match against the name only - e.g. ``foo.bar`` matches ``foo.bar``, ``foo.bar()``, and ``foo.bar(args, here)``, etc.
223223

224224
Example
225225
^^^^^^^

flake8_async/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@
3838

3939

4040
# CalVer: YY.month.patch, e.g. first release of July 2022 == "22.7.1"
41-
__version__ = "24.11.4"
41+
__version__ = "25.1.1"
4242

4343

4444
# taken from https://github.com/Zac-HD/shed

flake8_async/visitors/visitor91x.py

+131-21
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,14 @@
1-
"""Contains Visitor91X.
1+
"""Contains Visitor91X and Visitor124.
22
3-
910 looks for async functions without guaranteed checkpoints (or exceptions), and 911 does
4-
the same except for async iterables - while also requiring that they checkpoint between
5-
each yield.
3+
Visitor91X contains checks for
4+
* ASYNC100 cancel-scope-no-checkpoint
5+
* ASYNC910 async-function-no-checkpoint
6+
* ASYNC911 async-generator-no-checkpoint
7+
* ASYNC912 cancel-scope-no-guaranteed-checkpoint
8+
* ASYNC913 indefinite-loop-no-guaranteed-checkpoint
9+
10+
Visitor124 contains
11+
* ASYNC124 async-function-could-be-sync
612
"""
713

814
from __future__ import annotations
@@ -64,6 +70,92 @@ def func_empty_body(node: cst.FunctionDef) -> bool:
6470
)
6571

6672

73+
# this could've been implemented as part of visitor91x, but /shrug
74+
@error_class_cst
75+
class Visitor124(Flake8AsyncVisitor_cst):
76+
error_codes: Mapping[str, str] = {
77+
"ASYNC124": (
78+
"Async function with no `await` could be sync."
79+
" Async functions are more expensive to call."
80+
)
81+
}
82+
83+
def __init__(self, *args: Any, **kwargs: Any):
84+
super().__init__(*args, **kwargs)
85+
self.has_await = False
86+
self.in_class = False
87+
88+
def visit_ClassDef(self, node: cst.ClassDef):
89+
self.save_state(node, "in_class", copy=False)
90+
self.in_class = True
91+
92+
def leave_ClassDef(
93+
self, original_node: cst.ClassDef, updated_node: cst.ClassDef
94+
) -> cst.ClassDef:
95+
self.restore_state(original_node)
96+
return updated_node
97+
98+
# await in sync defs are not valid, but handling this will make ASYNC124
99+
# correctly pop up in parent func as if the child function was async
100+
def visit_FunctionDef(self, node: cst.FunctionDef):
101+
# default values are evaluated in parent scope
102+
# this visitor has no autofixes, so we can throw away return value
103+
_ = node.params.visit(self)
104+
105+
self.save_state(node, "has_await", "in_class", copy=False)
106+
107+
# ignore class methods
108+
self.has_await = self.in_class
109+
110+
# but not nested functions
111+
self.in_class = False
112+
113+
_ = node.body.visit(self)
114+
115+
# we've manually visited subnodes (that we care about).
116+
return False
117+
118+
def leave_FunctionDef(
119+
self, original_node: cst.FunctionDef, updated_node: cst.FunctionDef
120+
) -> cst.FunctionDef:
121+
if (
122+
original_node.asynchronous is not None
123+
and not self.has_await
124+
and not func_empty_body(original_node)
125+
and not func_has_decorator(original_node, "overload")
126+
# skip functions with @fixture and params since they may be relying
127+
# on async fixtures.
128+
and not (
129+
original_node.params.params
130+
and func_has_decorator(original_node, "fixture")
131+
)
132+
# ignore functions with no_checkpoint_warning_decorators
133+
and not fnmatch_qualified_name_cst(
134+
original_node.decorators, *self.options.no_checkpoint_warning_decorators
135+
)
136+
):
137+
self.error(original_node)
138+
self.restore_state(original_node)
139+
return updated_node
140+
141+
def visit_Await(self, node: cst.Await):
142+
self.has_await = True
143+
144+
def visit_With(self, node: cst.With | cst.For | cst.CompFor):
145+
if node.asynchronous is not None:
146+
self.has_await = True
147+
148+
visit_For = visit_With
149+
visit_CompFor = visit_With
150+
151+
# The generator target will be immediately evaluated, but the other
152+
# elements will not be evaluated at the point of defining the GenExp.
153+
# To consume those needs an explicit syntactic checkpoint
154+
def visit_GeneratorExp(self, node: cst.GeneratorExp):
155+
node.for_in.iter.visit(self)
156+
return False
157+
158+
67159
@dataclass
68160
class LoopState:
69161
infinite_loop: bool = False
@@ -275,7 +367,6 @@ class Visitor91X(Flake8AsyncVisitor_cst, CommonVisitors):
275367
def __init__(self, *args: Any, **kwargs: Any):
276368
super().__init__(*args, **kwargs)
277369
self.has_yield = False
278-
self.safe_decorator = False
279370
self.async_function = False
280371
self.uncheckpointed_statements: set[Statement] = set()
281372
self.comp_unknown = False
@@ -291,6 +382,9 @@ def __init__(self, *args: Any, **kwargs: Any):
291382
# --exception-suppress-context-manager
292383
self.suppress_imported_as: list[str] = []
293384

385+
# used to transfer new body between visit_FunctionDef and leave_FunctionDef
386+
self.new_body: cst.BaseSuite | None = None
387+
294388
def should_autofix(self, node: cst.CSTNode, code: str | None = None) -> bool:
295389
if code is None: # pragma: no branch
296390
code = "ASYNC911" if self.has_yield else "ASYNC910"
@@ -349,6 +443,10 @@ def visit_ImportFrom(self, node: cst.ImportFrom) -> None:
349443
return
350444

351445
def visit_FunctionDef(self, node: cst.FunctionDef) -> bool:
446+
# `await` in default values happen in parent scope
447+
# we also know we don't ever modify parameters so we can ignore the return value
448+
_ = node.params.visit(self)
449+
352450
# don't lint functions whose bodies solely consist of pass or ellipsis
353451
# @overload functions are also guaranteed to be empty
354452
# we also ignore pytest fixtures
@@ -358,7 +456,6 @@ def visit_FunctionDef(self, node: cst.FunctionDef) -> bool:
358456
self.save_state(
359457
node,
360458
"has_yield",
361-
"safe_decorator",
362459
"async_function",
363460
"uncheckpointed_statements",
364461
# comp_unknown does not need to be saved
@@ -370,11 +467,11 @@ def visit_FunctionDef(self, node: cst.FunctionDef) -> bool:
370467
"suppress_imported_as", # a copy is saved, but state is not reset
371468
copy=True,
372469
)
373-
self.has_yield = self.safe_decorator = False
374470
self.uncheckpointed_statements = set()
471+
self.has_checkpoint_stack = []
472+
self.has_yield = False
375473
self.loop_state = LoopState()
376474
# try_state is reset upon entering try
377-
self.has_checkpoint_stack = []
378475
self.taskgroup_has_start_soon = {}
379476

380477
self.async_function = (
@@ -383,36 +480,49 @@ def visit_FunctionDef(self, node: cst.FunctionDef) -> bool:
383480
node.decorators, *self.options.no_checkpoint_warning_decorators
384481
)
385482
)
386-
if not self.async_function:
387-
# only visit subnodes if there is an async function defined inside
388-
# this should improve performance on codebases with many sync functions
389-
return any(m.findall(node, m.FunctionDef(asynchronous=m.Asynchronous())))
483+
# only visit subnodes if there is an async function defined inside
484+
# this should improve performance on codebases with many sync functions
485+
if not self.async_function and not any(
486+
m.findall(node, m.FunctionDef(asynchronous=m.Asynchronous()))
487+
):
488+
return False
390489

391490
pos = self.get_metadata(PositionProvider, node).start # type: ignore
392491
self.uncheckpointed_statements = {
393492
Statement("function definition", pos.line, pos.column) # type: ignore
394493
}
395-
return True
494+
495+
# visit body
496+
# we're not gonna get FlattenSentinel or RemovalSentinel
497+
self.new_body = cast(cst.BaseSuite, node.body.visit(self))
498+
499+
# we know that leave_FunctionDef for this FunctionDef will run immediately after
500+
# this function exits so we don't need to worry about save_state for new_body
501+
return False
396502

397503
def leave_FunctionDef(
398504
self, original_node: cst.FunctionDef, updated_node: cst.FunctionDef
399505
) -> cst.FunctionDef:
400506
if (
401-
self.async_function
507+
self.new_body is not None
508+
and self.async_function
402509
# updated_node does not have a position, so we must send original_node
403510
and self.check_function_exit(original_node)
404511
and self.should_autofix(original_node)
405-
and isinstance(updated_node.body, cst.IndentedBlock)
512+
and isinstance(self.new_body, cst.IndentedBlock)
406513
):
407514
# insert checkpoint at the end of body
408-
new_body = list(updated_node.body.body)
409-
new_body.append(self.checkpoint_statement())
410-
indentedblock = updated_node.body.with_changes(body=new_body)
411-
updated_node = updated_node.with_changes(body=indentedblock)
515+
new_body_block = list(self.new_body.body)
516+
new_body_block.append(self.checkpoint_statement())
517+
self.new_body = self.new_body.with_changes(body=new_body_block)
412518

413519
self.ensure_imported_library()
414520

521+
if self.new_body is not None:
522+
updated_node = updated_node.with_changes(body=self.new_body)
415523
self.restore_state(original_node)
524+
# reset self.new_body
525+
self.new_body = None
416526
return updated_node
417527

418528
# error if function exit/return/yields with uncheckpointed statements
@@ -1039,8 +1149,8 @@ def visit_CompFor(self, node: cst.CompFor):
10391149
return False
10401150

10411151
# The generator target will be immediately evaluated, but the other
1042-
# elements will be lazily evaluated as the generator is consumed so we don't
1043-
# visit them as any checkpoints in them are not guaranteed to execute.
1152+
# elements will not be evaluated at the point of defining the GenExp.
1153+
# To consume those needs an explicit syntactic checkpoint
10441154
def visit_GeneratorExp(self, node: cst.GeneratorExp):
10451155
node.for_in.iter.visit(self)
10461156
return False

tests/autofix_files/async910.py

+21-2
Original file line numberDiff line numberDiff line change
@@ -134,9 +134,12 @@ def foo_normal_func_1():
134134
def foo_normal_func_2(): ...
135135

136136

137-
# overload decorator
137+
# overload decorator is skipped
138+
# overload functions should always be empty, so the check is somewhat redundant,
139+
# but making one non-empty to check the logic.
138140
@overload
139-
async def foo_overload_1(_: bytes): ...
141+
async def foo_overload_1(_: bytes):
142+
raise NotImplementedError
140143

141144

142145
@typing.overload
@@ -616,3 +619,19 @@ async def fn_226(): # error: 0, "exit", Statement("function definition", lineno
616619
except Exception:
617620
pass
618621
await trio.lowlevel.checkpoint()
622+
623+
624+
# the await() is evaluated in the parent scope
625+
async def foo_default_value_await():
626+
async def bar( # error: 4, "exit", Statement("function definition", lineno)
627+
arg=await foo(),
628+
):
629+
print()
630+
await trio.lowlevel.checkpoint()
631+
632+
633+
async def foo_nested_empty_async():
634+
# this previously errored because leave_FunctionDef assumed a non-empty body
635+
async def bar(): ...
636+
637+
await foo()

tests/autofix_files/async910.py.diff

+12-1
Original file line numberDiff line numberDiff line change
@@ -213,8 +213,19 @@
213213

214214

215215
# Issue #226
216-
@@ x,3 x,4 @@
216+
@@ x,6 x,7 @@
217217
pass
218218
except Exception:
219219
pass
220220
+ await trio.lowlevel.checkpoint()
221+
222+
223+
# the await() is evaluated in the parent scope
224+
@@ x,6 x,7 @@
225+
arg=await foo(),
226+
):
227+
print()
228+
+ await trio.lowlevel.checkpoint()
229+
230+
231+
async def foo_nested_empty_async():

0 commit comments

Comments
 (0)