Rename futures to tasks

[milesgranger]

Closes #8028

Depends on dask/dask#10906

TODO: Before merge, drop commit setting this branch for dask dependency in environment files.

It was a bit of a gray area for where the renaming should stop when considering user interface, then I could easily get confused when we're calling something a Task externally, but Future internally. So, I wrote a script to rename everything. Simple regex-search-replace won't work AFAIK b/c we have plenty of legitimate "future" references w/ asyncio, concurrent.futures etc.

Here's the current iteration of the script:

import re
import pathlib
import hashlib


# These files tend to use future nomenclature outside of 'old' dask future references
# ie asyncio, tornado, concurrent.futures
IGNORE_FILES = [
    pathlib.Path("./distributed/comm/inproc.py"),
    pathlib.Path("./distributed/comm/tests/test_comms.py"),
    pathlib.Path("./distributed/comm/tests/test_ucx.py"),
    pathlib.Path("./distributed/deploy/spec.py"),
    pathlib.Path("./distributed/lock.py"),
    pathlib.Path("./distributed/multi_lock.py"),
    pathlib.Path("./distributed/protocol/tests/test_compression.py"),
    pathlib.Path("./distributed/tests/test_threadpoolexecutor.py"),
    pathlib.Path("./distributed/utils.py"),
    pathlib.Path("./distributed/utils_test.py"),
    pathlib.Path("./docs/make.bat"),
]


def rename(path: pathlib.Path):
    try:
        text = path.read_text()
    except Exception as exc:
        return
    guarded = (
        "__future__",
        "future_",  # escapes for actual future variable names
        "futures_",  # escapes for actual future variable names
        "latest/futures",  # docs url
        "Tornado futures",
        "a future task",
        "future tasks",
        "sleep_future",
        "gen.Future",
        "FutureWarning",
        "Tornado futures",
        "cancel_futures",
        "cf.Future",
        "cf_future",
        "asyncio.Future",
        "create_future",
        "future.html.j2",
        "_base.Future",
        "ensure_future",
        "c.f.Future",
        "Concurrent.futures",
        "concurrent.futures Future",
        "concurrent.futures.Future",
        "Future as ConcurrentFuture",
        "Future as TornadoFuture",
        "_concurrent_futures_thread",
        "_exit_future",
        "ConcurrentFuture",
        "TornadoFuture",
        "concurrent.futures",
        "DoneAndNotDoneFutures",
        "future release",
        "future version",
        "the future when",
        "future-proofing",
        "Future-proof",
        "deprecated in the future",
        "will change in the future",
        "In the future",
        "change in the future",
        "removed in the future.",
        "in the future",
        "future warning",
    )
    encoded = {k: hashlib.md5(k.encode()).hexdigest() for k in guarded}
    decoded = {v: k for k, v in encoded.items()}

    # Encode escape phrases
    for key, value in encoded.items():
        text = text.replace(key, value)

    # Perform renaming
    changes = (
        (r"\bFuture\b", "Task"),
        (r"\bFutureState\b", "TaskState"),
        (r"\bclass Future\b", "class Task"),
        (r"\bclass FutureState\b", "class TaskState"),
        (r"\bBaseActorFuture\b", "BaseActorTask"),
        (r"\bActorFuture\b", "ActorTask"),
        (r"\bEagerActorFuture\b", "EagerActorTask"),
        (r"\b_gather_future", "_gather_task"),
        (r"\bfuture\b", "task"),
        (r"\b_future\b", "_task"),
        (r"\bcf_future\b", "cf_task"),
        (r"\b_cascade_future\(", "_cascade_task("),
        (r"\b_wrap_future\(", "_wrap_task("),
        (r"\bnormalize_future\(", "normalize_task("),
    )
    text = text.replace("futures", "tasks").replace("Futures", "Tasks")
    text = text.replace("future", "task").replace("Future", "Task")
    for regex, sub in changes:
        text = re.sub(regex, sub, text)

    # Decode escaped phrases
    for key, value in decoded.items():
        text = text.replace(key, value)

    # manually applied renaming prior to renaming script run
    text = re.sub(r"\bfuture_\b", "future", text)
    text = re.sub(r"\bfutures_\b", "futures", text)

    path.write_text(text)


def crawl(dir: pathlib.Path):
    for path in dir.iterdir():
        if path.is_dir():
            yield from crawl(path)
        yield path


def rename_in_dir(dir: pathlib.Path):
    if not dir.exists():
        return
    for path in crawl(dir):
        if "changelog" in path.name.lower():
            continue
        if path in IGNORE_FILES:
            continue
        rename(path)


def main():
    rename_in_dir(pathlib.Path("./dask"))
    rename_in_dir(pathlib.Path("./distributed"))
    rename_in_dir(pathlib.Path("./docs"))


if __name__ == "__main__":
    main()

[mrocklin]

Cool. I'm excited to see this being taken on. Definitely a big project though!

I pointed out some cases where we're probably being over-zealous in naming. Additionally, I think that we should care about a smooth transition (like naming Future = Task for backwards compatibility for now) and probably also thinking about adding some redirects in dask docs (this is easy to do in conf.py).

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    27 files  ±0      27 suites  ±0   10h 7m 19s ⏱️ + 8m 53s
 3 965 tests ±0   3 852 ✅  - 1    109 💤 ±0  4 ❌ +1 
49 847 runs  ±0  47 548 ✅ ±0  2 295 💤  - 1  4 ❌ +1 

For more details on these failures, see this check.

Results for commit abab923. ± Comparison against base commit 4425516.

This pull request removes 27 and adds 27 tests. Note that renamed tests count towards both.

distributed.tests.test_actor ‑ test_future
distributed.tests.test_as_completed ‑ test_as_completed_with_non_futures
distributed.tests.test_client ‑ test_Future_exception
distributed.tests.test_client ‑ test_Future_exception_sync
distributed.tests.test_client ‑ test_Future_exception_sync_2
distributed.tests.test_client ‑ test_Future_release
distributed.tests.test_client ‑ test_Future_release_sync
distributed.tests.test_client ‑ test_await_future
distributed.tests.test_client ‑ test_call_stack_future
distributed.tests.test_client ‑ test_normalize_collection_with_released_futures
…

distributed.tests.test_actor ‑ test_task
distributed.tests.test_as_completed ‑ test_as_completed_with_non_tasks
distributed.tests.test_client ‑ test_Task_exception
distributed.tests.test_client ‑ test_Task_exception_sync
distributed.tests.test_client ‑ test_Task_exception_sync_2
distributed.tests.test_client ‑ test_Task_release
distributed.tests.test_client ‑ test_Task_release_sync
distributed.tests.test_client ‑ test_await_task
distributed.tests.test_client ‑ test_call_stack_task
distributed.tests.test_client ‑ test_normalize_collection_with_released_tasks
…

♻️ This comment has been updated with latest results.