System.Numerics or Stride.Math

[Doprez]

I wanted to open a tracked discussion on using System.Numerics instead of Strides created types.

• I am curious if there were any reasons before as to not using numerics in the first place and if so what were they?
• what would need to change if we did a full conversion in our libraries
• what would/could break in user projects right now
• will there be a major performance impact and is there an easy way to quickly benchmark before doing this work?
• other

As far as I understand it the only breaking change would be that users would need to add an extra using statement to their existing projects for .NETs math libraries.

[Doprez]

Adding comments from Eideren that went over this already but they are getting lost in the discord.

So, we can either:
Deprecate our math type, replace their usage in source outside of Stride.Core.Mathematics with dotnet's System.Numerics' type. Add implicit conversion from our types to System.Numerics to reduce breaking changes. Create a new static type Stride.Mathematics for operations that are not covered by dotnet's numerics API.
Ref parameters, overriden virtual methods, compound assignment, etc. will still create some breaking changes
System.Numerics doesn't have equivalent to a lot of our types, like BoundingBox, ..Frustum, Ray, ... nor explicit types for other vector data types like half2-3, int2-3 ... This is going to be a bit confusing as now users would have to fetch vectors from dotnet's namespace and other datatypes from our math namespace.
We don't have full control over the type, meaning that implementing more operators and conversions is impossible for the former and fairly hard for the latter as long as dotnet doesn't support extensions for those function types.
Same as above but instead of relying on dotnet's we create a new one which wraps around dotnet's type internally.
Breaking changes isn't any better
No confusing namespace
We keep control over the whole type
More work on our part to create this one, cover the numerics api and keep up to date with changes dotnet might introduce
Keep our type, use dotnet's accelerated operations under the hood, break ref params anyway
Breaking changes are limited to ref params.
Larger breaking changes like below are harder to justify.
 
In #1 & #2 I think it would be a good idea to also adopt the way dotnet's System.Math is designed; Math.Round has overloads for int, float, double etc. under the same static type, instead of having Vector3.Round, Vector4.Round, etc. Also less confusing for operations that take different data types like RotationQuaternion(Quaternion rotation, out Matrix result) - should that be under Matrix or Quaternion ... and finally merging all of that with the not very well known MathUtil to help users discover helpers for angle conversions and such.

[Doprez]

Another comment from Ethereal to keep in mind

Regarding the math types, I think for Stride 4.2 we shouldn't do any major rewrite yet.
The .NET team has created many useful types and functions, but support for generic Vector2<T> where T can be float, double, Half, etc., is postponed to .NET 9 I believe. Those would be the more interesting targets to port the Stride math types to.
The Silk.NET team is also having a discussion right now about how to enhance their math library, with many interesting comments from Tanner Gooding. You can check [here](https://github.com/dotnet/Silk.NET/pull/1760#discussion_r1375156663).

referenced link.

[Doprez]

final thing I found that we should keep in mind is Silks ongoing work for 3.0. They are apparently dealing with this exact issue.
dotnet/Silk.NET#1727

[Perksey]

Yes it’s always been our intention for Silk.NET.Maths to be a “gap filler” and to always prefer System.Numerics where possible. By the sounds of this issue, Silk.NET.Maths as the same intentions as that of Stride.Mathematics discussed here.

[Kryptos-FR]

I am curious if there were any reasons before as to not using numerics in the first place and if so what were they?

Yes there was a big one. Those types just didn't exist when the engine was first being worked on. And after they were introduced they had at first slightly worse performance.

[ericwj]

With regard to what Eideren said, I think it would be beneficial in many places to replace some implementations in Stride.Math (method bodies) delegating as much as possible to usually well-optimized implementations in System.Numerics and review Stride.Math for opportunities to use intrinsics. For example converting RGBA to BGRA can be done in 2 instructions (BSWAP, ROL) or less (vectorized) and in some places what I remember it is done (or used to be done when I was working on a PR) in some 54.

One thing I recommend is marking the overloads with ref parameters obsolete. When I looked at it, it turns out they are often much slower and knowing how parameter passing works it is mostly pointless to pass a writable ref anyway. For parameters that fit a register or that might be broken into pieces and passed in more than one register, it might force them to stay on the stack.

By marking the obsoletions with some specific diagnostic ID, subsequently suppressing that warning in each project, people can go about their business without getting swamped with warnings while some issue tracks removing the suppression on a per-project basis, starting with the most specific projects and working backwards to the core Stride projects.

[Doprez]

Another discussion found thanks to Eideren that will need to be considered for matrices #291

[Doprez]

Another natural occurrence of system.Numerics ideas #2238 (comment)

I would wonder if FieldOffset could also be used as a conversion between the Stride Matrix and System.Numerics.Matrix4x4 to map between row major and column major mappings?

[JeroMiya]

For what it's worth, I'd like to provide some feedback, as this is something I'd really like to see implemented (using System.Numerics where possible). I feel it would be an overall benefit to the project in the long term. Of course, I'm not a game engine programmer, and I'm aware that there are limitations to System.Numerics, but overall I think the benefits outweigh the drawbacks.

Backwards compatibility:
Although I may currently be coming from the perspective of a new developer without a lot of legacy code to maintain, I feel confident that if I did, I would still consider backward compatibility a non-issue. The changes would likely be superficial.

The issue of not controlling the types, vs the custom types in the engine:
I would prefer using the System.Numerics types as-is, vs custom types that internally wrap the types. Being able to use those types in userland code is one of the main benefits to migrating to System.Numerics, so it would be a shame to lose that. Where necessary, my preference would be method overloads (this seems to mostly be the strategy of Silk.Net as far as I can tell), static utility methods, and if wrapping is absolutely necessary - then maybe hiding the wrapper classes as internal to the engine could be done. One benefit of method overloads vs wrapper classes is that we could, for example, support new types being added to System.Numerics while either continuing to support the old custom types or we could more gracefully deprecate them over a longer period.

Implicit Conversions:
My preference would be explicit conversions over implicit. Technically, an implicit conversion is valid if the conversion is lossless and one-to-one, but for game development I personally put "conversion has a performance impact" under the threshold moving the needle from implicit to explicit. And again, using method overloads instead of wrapper classes or custom types would make either implicit or explicit conversions less necessary for most code. That being said, I don't feel strongly on this and implicit conversions wouldn't be the worst thing.

Math types that don't currently exist in System.Numerics:
I'm ok with filling in the gaps with custom types where necessary. I think this is the Silk.Net strategy and it makes practical sense. But, for types like Vector3/Vector4/etc... that do have types in System.Numerics, my preference is to use those types directly (not wrapped in a custom type). The "different types from different namespaces" isn't a big issue. For my own projects, I use global using statements so I would only have to add one additional using statement in one file per csproj. Not a huge deal. And of course if System.Numerics adds the other types, we might be able to add method overloads to support them over time without breaking backward compatibility.

Consolidating math utils under one MathUtils static class:
I agree with this suggestion. Especially for new users, typing something like MathUtils. in the code editor and taking a look at the autocomplete list will help them find the static methods they need.

tannergooding's comment on public fields:
Hard agree here - public properties highly preferred over public fields. We're not Unity.

[Doprez]

A very late update to this but tracking some of the thoughts/updates here.

Backwards compatibility:

This should now be in place with the implicit/explicit conversions. The old Stride types still exist so this shouldn't be an issue currently.

The issue of not controlling the types, vs the custom types in the engine:

The internal wrappers are currently in the works, main issue so far is how System.Numerics handles nullability compared to Stride which causes some issues. I think it would make sense to keep Stride.Core.Math as the main library for users to use and then internally have things use System.Numerics for the performance benefits that seem to keep coming in each .NET iteration.

Implicit Conversions:

This should be resolved. The main types like Vector3 and Quaternion seem to be free thanks to Unsafe conversions. The main exception that I know of is Matrix due to the required transpose required to convert properly. I havent check Int2, Int3, etc but those are currently marked as explicit.

[MechWarrior99]

Just to chime in and add my two cents. I am pretty heavily against switching to System.Numerics, for a couple of reasons.

Having math types spread across multiple namespaces causes confusion, especially for newer users. And it can be hard to remember which namespace a type is in. Of course this can be mitigated with default or global namespace. But to me it just looks worse (very subjective of course). And also doesn't solve the issue where some math types and methods would be in Stride's documentation while others would not, seemingly at 'random'.

More niche types, like line, ray, bounds, etc. may be (and are) added to System.Numerics, but may not function as we want or need. Which would result in multiple types with similar names and uses. Or having to change Stride to fit. Again, adding potential for confusion. And if they are added later on to System.Numerics (I know they are planning on adding some), there will be naming conflics requiring us to change our code and cause issues for users.

Not controlling the type means we cannot add or change methods as needed. This results in either making extension methods (which are different than the normal static method pattern), or have a seperate additional class for them, like Vector3Util. This means having to keep track of where certain methods are. Another option is to just have a singular static Maths class that contains the math methods for all types. But many of the same methods would be contained on the System.Numerics types, which again, can cause confusion as to why sme types have some static methods and why others don't and what the difference is between the ones on the type and in the math class.

The math for Stride's types are different from the System math. Around handling of NaN, and infinite values. And especially around the matrix/matrix4x4. The numerics matrix4x4x also has some methods that are not very relevent for Stride, again, adding potential confusion.

There are some benefits of course. Being able to more easily work with other libraries is great, and in some cases we can see decent performance increases without having to do any extra work.

It isn't like you can't work around any of these things of course. But to me, the trade offs in usability, and extensability out weigh the benefits. And I can't help but feel that giving up control of the math types will come back to bite us in the end.

But that is just my opinion, thanks for opening this discussion on the topic!

[Doprez]

Having math types spread across multiple namespaces causes confusion...

I guess the only reason this would be an issue is due to System.Numerics not having all of the required types? Otherwise Vectors, matrices, quaternions, etc would be full Numerics not needing Stride namespaces.

More niche types, like line, ray, bounds, etc. may be (and are) added to System.Numerics, but may not function as we want or need...

Would this be an issue with System.Numerics or with Strides naming? I know there was specifically an issue with System.Numerics Matrix4x4 being column major compared to Stride being row major so it definitely would not hurt to investigate deeper.

Not controlling the type means we cannot add or change methods as needed...

Hopefully this will not be the case in newer .NET with the ability to add implicit extensions to external types. Although with current .NET I agree and there are a lot of places where an extension method would be a breaking change instead of an internal type. Im also not sure how common it would be to change core math functionality, i would expect it to be the same across most scenarios but again worth investigating requirements just to be sure.

The math for Stride's types are different from the System math. Around handling of NaN, and infinite values...

I definitely found this when trying to change more than just the types. It became an issue with the System.Numerics math methods specifically returning null instead of zero like you are saying.

There are some benefits of course. Being able to more easily work with other libraries is great, and in some cases we can see decent performance increases without having to do any extra work.

It isn't like you can't work around any of these things of course. But to me, the trade offs in usability, and extensability out weigh the benefits. And I can't help but feel that giving up control of the math types will come back to bite us in the end.

It may be worth stopping here if this is a common concern, We have already added implicit conversions to System.Numerics which now allows Stride math types to work with other Numerics libraries without changing any of the core Stride functionality. In terms of cross compatibility this is probably enough and still allows for the people with more performance focused goals to use the System.Numerics types with Stride handling most of the conversion (There may be some oddities due to the null handling but overall seemed to work for the most part in my cases).

But that is just my opinion, thanks for opening this discussion on the topic!

Thank you, for commenting! Its always useful yo have multiple opinions and thoughts, especially on potentially breaking changes.

[MechWarrior99]

I guess the only reason this would be an issue is due to System.Numerics not having all of the required types?

Yeah, it only has Vector2/3/4, Matrix4x4, Matrix2x3, Quaternion, Plane ( and a few others). Things like Int2/3/4, Double2/3/4, Rectangle, Size2, Ray, BoundingBox, etc. are all in Stride.Core.Mathematics, so quite a few (though most are less used).

Would this be an issue with System.Numerics or with Strides naming?

It is speculative, and not really an issue on either part, simply a natural by product. If we have System.Numerics as a 'default' namespace then if they add say a type called BoundingBox, any current uses of the Stride.Core.Mathematics will cause an compile error. Also, the System.Numerics BoundingBox could be defined by a position and extents like Unity's, instead of a Minimum and Maximum position like Stride's, making it not a straight forward replacement. Also, Stride's BoundingBox implements interfaces like IInterstableWithRay, which the System.Numerics would of course not implement.

It is hypothetical, but in my opinion a fairly realistic scenario. I've talked on the C# discord with Tanner Gooding and they have mentioned wanting to expand the geometric math types in System.Numerics.

[..] allows Stride math types to work with other Numerics libraries without changing any of the core Stride functionality.

Oh this is nice, and feels like it handles most needs without the drawbacks.

[Kryptos-FR]

I liked the idea of having our types use the Numerical one as implementation détail. That way from an API perspective and for our users only our types are visible. But because we use the Numerical one "under the hood", we can benefit from hardware acceleration, as much as possible.

[JeroMiya]

Just want to throw in my own friendly feedback:

I am, and have been for some time, VERY strongly in favor of .net frameworks, libraries, and game engines standardizing on using System.Numerics where possible (both internally and throughout the "public" APIs), and expanding use over time as the scope of System.Numerics broadens. I often hear arguments against it, but I don't think they are sufficient to block what I think is a better overall potential state for the .net ecosystem to be in.

One argument against standardizing stems from the worry that it may be confusing it is for developers to use types in System.Numerics and types in another namespace, for instance for types and utilities that System.Numerics doesn't yet support. I think this argument doesn't consider the big picture: the current status quo is that EVERY framework, library, and game engine has its own set of nearly identical types for the same set of core functionality. Meaning, the end-user developer already has to manage an even worse situation than the one the above argument worries about. Implicit conversions are nice, but they're not a clean workaround. At least when System.Numerics types are used consistently, the core set of types are the same across engines and third party libraries. So developer experience in one can carry over to the other.

Another argument is with potential conflicts and potential breaking changes when System.Numerics adds new types that previously we had to implement in the engine, as they were previously not available in System.Numerics. I would clarify that it is not in fact a breaking change for one assembly to add a new type in a given namespace, even when an existing type has the same name but in another namespace. The fully qualified name of the new type will be different than that of the existing type, as the namespace is going to be different (and the assembly, too). Thus, assemblies will remain binary compatible.

Although this may result in a new compilation error if a C# file has using statements for both namespaces, I think this exactly the type of issue that namespaces are meant to solve. And, it should be said, this is not exactly a new problem, or indeed unique to System.Numerics or any other type in the base class library. New types can be added to the BCL at any time, often with the same names as existing types in third party libraries. This is easily managed as it always has been.

Speaking of the BCL, I can't help but notice that most of the arguments against standardizing on System.Numerics really could be said about any other module of the BCL or any of the extension packages - indeed, as an example, it is true that there is functionality in System.DateTime and System.DateTimeOffset that is missing, and there are third party libraries that fill those gaps, but I don't think anyone is arguing that we should have a custom DateTime and TimeSpan library for every game engine, physics engine, asset import library, and graphics library binding "just in case" it might be confusing for developers to have to use BCL types along with custom types that fill in those gaps. As I point out in the first paragraph above, I actually think requiring developers to learn N number of DateTime or TimeSpan types (or in this case Vector3, Matrix4x4, etc...) for every library and game engine is a worse situation to be in.

As for my own suggestions, I would council the following:

• I would strongly prefer plain static methods to extend functionality of System.Numerics types, over extension methods. As people have pointed out, extension methods have some ergonomic, readability, and backwards compatibility issues that make them a better fit for fluent APIs designed to be extended (for example IServiceCollection), vs types that are meant to be standalone (for example Vector3).
• Use System.Numerics types both internally and externally. Additional overloads for custom types are fine.

I would like to understand better the mentioned differences in semantics between System.Numerics and the existing Stride math library. Some differences in handling of NaN and Infinity values was mentioned, but I think this needs more discussion and elaboration. Is System.Numerics doing something non-sensible in this regard? Is Stride following conventions or semantics that are common in game engines but uncommon in general purpose math libraries? This seems like a concerning situation.