|
| 1 | +# Techniques for fixing inference problems |
| 2 | + |
| 3 | +Here we assume you've dug into your code with a tool like Cthulhu, and want to know how to fix some of the problems that you discover. Below is a collection of specific cases and some tricks for handling them. |
| 4 | + |
| 5 | +Note that there is also a [tutorial on fixing inference](@ref inferrability) that delves into advanced topics. |
| 6 | + |
| 7 | +## Adding type annotations |
| 8 | + |
| 9 | +### Using concrete types |
| 10 | + |
| 11 | +Defining variables like `list = []` can be convenient, but it creates a `list` of type `Vector{Any}`. This prevents inference from knowing the type of items extracted from `list`. Using `list = String[]` for a container of strings, etc., is an excellent fix. When in doubt, check the type with `isconcretetype`: a common mistake is to think that `list_of_lists = Array{Int}[]` gives you a vector-of-vectors, but |
| 12 | + |
| 13 | +```jldoctest |
| 14 | +julia> isconcretetype(Array{Int}) |
| 15 | +false |
| 16 | +``` |
| 17 | + |
| 18 | +reminds you that `Array` requires a second parameter indicating the dimensionality of the array. (Or use `list_of_lists = Vector{Int}[]` instead, as `Vector{Int} === Array{Int, 1}`.) |
| 19 | + |
| 20 | +Many valuable tips can be found among [Julia's performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/), and readers are encouraged to consult that page. |
| 21 | + |
| 22 | +### Working with non-concrete types |
| 23 | + |
| 24 | +In cases where invalidations occur, but you can't use concrete types (there are indeed many valid uses of `Vector{Any}`), |
| 25 | +you can often prevent the invalidation using some additional knowledge. |
| 26 | +One common example is extracting information from an [`IOContext`](https://docs.julialang.org/en/v1/manual/networking-and-streams/#IO-Output-Contextual-Properties-1) structure, which is roughly defined as |
| 27 | + |
| 28 | +```julia |
| 29 | +struct IOContext{IO_t <: IO} <: AbstractPipe |
| 30 | + io::IO_t |
| 31 | + dict::ImmutableDict{Symbol, Any} |
| 32 | +end |
| 33 | +``` |
| 34 | + |
| 35 | +There are good reasons that `dict` uses a value-type of `Any`, but that makes it impossible for the compiler to infer the type of any object looked up in an `IOContext`. |
| 36 | +Fortunately, you can help! |
| 37 | +For example, the documentation specifies that the `:color` setting should be a `Bool`, and since it appears in documentation it's something we can safely enforce. |
| 38 | +Changing |
| 39 | + |
| 40 | +``` |
| 41 | +iscolor = get(io, :color, false) |
| 42 | +``` |
| 43 | + |
| 44 | +to |
| 45 | + |
| 46 | +``` |
| 47 | +iscolor = get(io, :color, false)::Bool # assert that the rhs is Bool-valued |
| 48 | +``` |
| 49 | + |
| 50 | +will throw an error if it isn't a `Bool`, and this allows the compiler to take advantage of the type being known in subsequent operations. |
| 51 | + |
| 52 | +If the return type is one of a small number of possibilities (generally three or fewer), you can annotate the return type with `Union{...}`. This is generally advantageous only when the intersection of what inference already knows about the types of a variable and the types in the `Union` results in an concrete type. |
| 53 | + |
| 54 | +As a more detailed example, suppose you're writing code that parses Julia's `Expr` type: |
| 55 | + |
| 56 | +```julia |
| 57 | +julia> ex = :(Array{Float32,3}) |
| 58 | +:(Array{Float32, 3}) |
| 59 | + |
| 60 | +julia> dump(ex) |
| 61 | +Expr |
| 62 | + head: Symbol curly |
| 63 | + args: Vector{Any(3,)) |
| 64 | + 1: Symbol Array |
| 65 | + 2: Symbol Float32 |
| 66 | + 3: Int64 3 |
| 67 | +``` |
| 68 | +
|
| 69 | +`ex.args` is a `Vector{Any}`. |
| 70 | +However, for a `:curly` expression only certain types will be found among the arguments; you could write key portions of your code as |
| 71 | +
|
| 72 | +```julia |
| 73 | +a = ex.args[2] |
| 74 | +if a isa Symbol |
| 75 | + # inside this block, Julia knows `a` is a Symbol, and so methods called on `a` will be resistant to invalidation |
| 76 | + foo(a) |
| 77 | +elseif a isa Expr && length((a::Expr).args) > 2 |
| 78 | + a::Expr # sometimes you have to help inference by adding a type-assert |
| 79 | + x = bar(a) # `bar` is now resistant to invalidation |
| 80 | +elseif a isa Integer |
| 81 | + # even though you've not made this fully-inferrable, you've at least reduced the scope for invalidations |
| 82 | + # by limiting the subset of `foobar` methods that might be called |
| 83 | + y = foobar(a) |
| 84 | +end |
| 85 | +``` |
| 86 | +
|
| 87 | +Other tricks include replacing broadcasting on `v::Vector{Any}` with `Base.mapany(f, v)`--`mapany` avoids trying to narrow the type of `f(v[i])` and just assumes it will be `Any`, thereby avoiding invalidations of many `convert` methods. |
| 88 | +
|
| 89 | +Adding type-assertions and fixing inference problems are the most common approaches for fixing invalidations. |
| 90 | +You can discover these manually, but using Cthulhu is highly recommended. |
| 91 | +
|
| 92 | +## Inferrable field access for abstract types |
| 93 | +
|
| 94 | +When invalidations happen for methods that manipulate fields of abstract types, often there is a simple solution: create an "interface" for the abstract type specifying that certain fields must have certain types. |
| 95 | +Here's an example: |
| 96 | +
|
| 97 | +``` |
| 98 | +abstract type AbstractDisplay end |
| 99 | + |
| 100 | +struct Monitor <: AbstractDisplay |
| 101 | + height::Int |
| 102 | + width::Int |
| 103 | + maker::String |
| 104 | +end |
| 105 | + |
| 106 | +struct Phone <: AbstractDisplay |
| 107 | + height::Int |
| 108 | + width::Int |
| 109 | + maker::Symbol |
| 110 | +end |
| 111 | + |
| 112 | +function Base.show(@nospecialize(d::AbstractDisplay), x) |
| 113 | + str = string(x) |
| 114 | + w = d.width |
| 115 | + if length(str) > w # do we have to truncate to fit the display width? |
| 116 | + ... |
| 117 | +``` |
| 118 | +
|
| 119 | +In this `show` method, we've deliberately chosen to prevent specialization on the specific type of `AbstractDisplay` (to reduce the total number of times we have to compile this method). |
| 120 | +As a consequence, Julia's inference may not realize that `d.width` returns an `Int`. |
| 121 | +
|
| 122 | +Fortunately, you can help by defining an interface for generic `AbstractDisplay` objects: |
| 123 | +
|
| 124 | +``` |
| 125 | +function Base.getproperty(d::AbstractDisplay, name::Symbol) |
| 126 | + if name === :height |
| 127 | + return getfield(d, :height)::Int |
| 128 | + elseif name === :width |
| 129 | + return getfield(d, :width)::Int |
| 130 | + elseif name === :maker |
| 131 | + return getfield(d, :maker)::Union{String,Symbol} |
| 132 | + end |
| 133 | + return getfield(d, name) |
| 134 | +end |
| 135 | +``` |
| 136 | +
|
| 137 | +Julia's [constant propagation](https://en.wikipedia.org/wiki/Constant_folding) will ensure that most accesses of those fields will be determined at compile-time, so this simple change robustly fixes many inference problems. |
| 138 | +
|
| 139 | +## Fixing `Core.Box` |
| 140 | +
|
| 141 | +[Julia issue 15276](https://github.com/JuliaLang/julia/issues/15276) is one of the more surprising forms of inference failure; it is the most common cause of a `Core.Box` annotation. |
| 142 | +If other variables depend on the `Box`ed variable, then a single `Core.Box` can lead to widespread inference problems. |
| 143 | +For this reason, these are also among the first inference problems you should tackle. |
| 144 | +
|
| 145 | +Read [this explanation of why this happens and what you can do to fix it](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured). |
| 146 | +If you are directed to find `Core.Box` inference triggers via [`suggest`](@ref), you may need to explore around the call site a bit-- |
| 147 | +the inference trigger may be in the closure itself, but the fix needs to go in the method that creates the closure. |
| 148 | +
|
| 149 | +Use of `ascend` is highly recommended for fixing `Core.Box` inference failures. |
| 150 | +
|
| 151 | +## Handling edge cases |
| 152 | +
|
| 153 | +You can sometimes get invalidations from failing to handle "formal" possibilities. |
| 154 | +For example, operations with regular expressions might return a `Union{Nothing, RegexMatch}`. |
| 155 | +You can sometimes get poor type inference by writing code that fails to take account of the possibility that `nothing` might be returned. |
| 156 | +For example, a comprehension |
| 157 | +
|
| 158 | +```julia |
| 159 | +ms = [m.match for m in match.((rex,), my_strings)] |
| 160 | +``` |
| 161 | +might be replaced with |
| 162 | +```julia |
| 163 | +ms = [m.match for m in match.((rex,), my_strings) if m !== nothing] |
| 164 | +``` |
| 165 | +and return a better-typed result. |
0 commit comments