Long vector test plan #421

alsepkow · 2025-03-07T20:43:38Z

.

damyanp

Looks good so far.

We try and format these specs to 80-columns to make them easier to review in source form.

Are the tables of operations a list of all the things that'll need tests added or is that still being determined?

proposals/infra/INF-0005-Long-Vector-TestPlan.md

damyanp · 2025-03-07T21:49:37Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+        - For tests where we only add new test cases we do not need to update HLK test GUIDs.
+        - If we want to add a new HLK requirement, then we will probably want to create new tests which consume that requirement. Individual test cases can not have individual HLK requirements.
+        - The existing HLK reqs we have look pretty dated. "Device.Graphics.WDDM27.AdapterRender.D3D12.DXILCore.ShaderModel65.CoreRequirement". Add a requirement for SM 6.9? We're requiring Long Vectort support for SM 6.9
+    3. Do the work in two 'phases'? Add the tests. Then do the HLK side.


I think we should build up a plan that involves multiple deliveries to the IHVs. If we can identify a set of high-priority tests (eg those needed by a particular demo) we could try and get those out first to support IHVs building drivers for the preview release.

I was thinking about this and if IHVs are OK with it I think the easiest way is to just share stand-alone test binaries/collateral. Can you help me sync up with the right people to figure out what works for IHVs? What have we done in the past?

Yup, we can sort that out, but I'm more interested in what and when we'll deliver these rather than the mechanism used to do it.

Actually, given that the DXC repo is public we should be able to just share build/run instructions for the tests once checked in?

alsepkow · 2025-03-10T23:26:46Z

Looks good so far.

We try and format these specs to 80-columns to make them easier to review in source form.

Are the tables of operations a list of all the things that'll need tests added or is that still being determined?

Will update the formatting. The intrinsics in the table are all of the explicitly listed intrinsics in the hlsl long vector. I think some operators are missing from this table. I'll look to get everything added so we have an exhaustive list.

damyanp · 2025-03-13T20:14:26Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+
+# Vector Sizes to test
+
+I don't think there are any particularly interesting vector sizes to test. So I


I don't think there are any particularly interesting vector sizes to test.

I think we might be able to come up with some.

For example, I think vector<half, 7>, vector<half, 8> and vector<half, 9> may be interesting because they cover the 128-bit boundary that has been in place most of the time before and could be interesting for load/stores as well as how they work in groupshared.

In addition to Damyan's suggestions, what comes to mind are:

vec5s - one above the previous vector limit. Also for the 128 boundary case in the case of 32 bit values that Damyan mentioned.

vec16 - previously only appeared as matrices, which might introduce some complications

vec17 - bigger than any vector previously possible

double3, uint64_t3 - another way to just exceed the 128 boundary, but with the added complexity of 64-bit types.

1024 for float, half, double, and int64 - It's the maximum size after all.

Thanks, I've updated to reflect these test cases.

Can either of you help point me in a direction to better understand how this could be interesting when testing load/stores and how they work in groupshared?

damyanp · 2025-03-13T20:16:04Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+'ExecTests' source code in the DXC repo. There is a script in the WinTools repo
+which generates and annotates the HLK tests.
+
+There are three test categories we are concerned with:


Do we need special tests for groupshared?

Are there any other types of storage that might be interesting?

Probably, I was thinking about this and I need to learn/explore more. I suspect I'm actually missing details on test case granularity. Does testing across data types matter? Do we care to verify the output of operations for correctness?

We should probably be testing for correctness, yes. Sizes, especially around interesting alignment cases, are probably interesting. It might be interesting to build cases that could conceivably generate overruns and check for them. For example, most GPUs will operate on at least 32-bits at once, so if you have 16-bit values then what happens if you have an odd number of elements - could it accidentally overwrite the next variable if it is assuming alignment?

And so the size of a type, and the element count, gets interesting. I expect you could come up with some interesting boundary cases and reuse that across all the types without having to think hard about each combination individually.

proposals/infra/INF-0005-Long-Vector-TestPlan.md

pow2clk

Looks good! Just trying to add detail and answer questions where I can.

proposals/infra/INF-0005-Long-Vector-TestPlan.md

pow2clk · 2025-03-17T16:54:18Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+     - These are the test cases for the 'basic' HLSL operators listed in the
+     HLSL operators table below.
+     - We have added some test cases for these with the long vectors work, but
+     we will want to make sure we add HLK coverage for all of these.


If you're saying that the HLK tests will cover the same operations as the added test cases, that's useful, but they don't add anything to HLK coverage directly as this seems to imply.

I'm probably confused here. I was assuming most, if not all, operators would map to some LLVM or DXIL operator? If that's true then wouldn't it make sense to have test cases in the HLK that use those operators with vector arguments?

pow2clk · 2025-03-17T17:00:42Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+
+# Vector Sizes to test
+
+I don't think there are any particularly interesting vector sizes to test. So I


In addition to Damyan's suggestions, what comes to mind are:

vec5s - one above the previous vector limit. Also for the 128 boundary case in the case of 32 bit values that Damyan mentioned.

vec16 - previously only appeared as matrices, which might introduce some complications

vec17 - bigger than any vector previously possible

double3, uint64_t3 - another way to just exceed the 128 boundary, but with the added complexity of 64-bit types.

1024 for float, half, double, and int64 - It's the maximum size after all.

proposals/infra/INF-0005-Long-Vector-TestPlan.md

pow2clk · 2025-03-17T17:23:26Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+| asint              | Emulated | |
+| asint16            | Emulated | |
+| asuint             | DXIL::OpCode::SplitDouble | |
+| asuint16           | Emulated | |


I think I'm mostly confused by the differing levels of detail in the "emulated" entries. Maybe they are just placeholders for eventual expansion. That's fine if so, but ultimately we want a list of DXOps and LLVM operators that we need to test. Showing how they map to the HLSL intrinsics is showing your work and that's fine, but most useful would be the list of DXIL::OpCode::* and LLVM operators.

'emulated' was essentially a placeholder. When I first started going through the list of HLSL intrinsics I was taking some notes on the LLVM/DXIL ops that I could see it distilled down to. I got stuck on several and gave up and went with 'emulated' for now.

I'll go back through everything that is marked as 'emulated' so we have an inventory of the operators used.

damyanp

Looking good - there's still some open issues.

I'm also keen to see a prioritization of which tests will be implemented first.

These operations are good candidates for high-priority tests I think:

Initializing a vector with another.
Multiply all components of a vector with a scalar value
Add all components of a vector with a scalar value
Clamp all components of a vector to the range [c, t]
Component wise minimum between 2 vectors
Component wise maximum between 2 vectors
Component wise multiply between 2 vectors
Component wise add between 2 vectors
Subscript access, vec[i] = c

damyanp · 2025-03-20T16:26:30Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+    their mapped DXIL OpCodes. We just need coverage for each DXIL OpCode. If
+    the DXIL OpCode column lists 'emulated' then this means that there are
+    multiple LLVM/DXIL Ops used to compute the intrinsic. We will need to do an


If the DXIL OpCode column lists 'emulated' then this means that there are multiple LLVM/DXIL Ops used to compute the intrinsic.

I think you've changed how this appears in the document now?

damyanp · 2025-03-20T16:27:46Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+     vectors across buffer types. __TODO: Add details about which types of
+     buffers to test and why.__


TODO: Add details about which types of buffers to test and why

This comment here to remind us that there's a TODO. If we want to complete the PR without doing the TODO then we should get a follow-up issue filed.

Does this include vector load/stores as well?

I'd prefer to fill in the details before completing this PR.

proposals/infra/INF-0005-Long-Vector-TestPlan.md

damyanp · 2025-03-20T16:36:02Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+| atan2     | Atan | CreateFDiv, CreateFAdd, CreateFSub, CreateFCmpOLT,
+||| CreateFCpmOEQ, CreateFCmpOGE, CreateFCmpOLT, CreateAnd, CreateSelect | |


I'm guessing that these "CreateXXX" function names came from the C++ code. The LLVM instructions are things like fdiv, fadd, fcmp, and, select etc.

I also note that if I try atan2 in godbolt it doesn't actually generate fsub. https://godbolt.org/z/7xqz6448P

Yes, they're the C++ helper functions. I was intending to distill down further on a subsequent iteration. Wanted to sanity check that they do what I would expect (always return the llvm instruction in the name).

I'll have to peel a little more in TranslateAtan2, for this 'inventory' I went through the various Translate* functions in HLSLOperationLower and noted the possible operations and instructions. For TranslateAtan2 it looks like FSub is always used, so it must be optimized or simplified out along the way?

Asked for some help. The FSub is optimized out to normalize to FAdd. If you pass '-O0' instead you'll see that the FSub is there.

But this makes me wonder about the right testing approach to ensure coverage. Given that the goal is to test that DXIL Ops and LLVM Instructions are functional with vectors, maybe we want to disable optimizations for the test cases?

proposals/infra/INF-0005-Long-Vector-TestPlan.md

anupamachandra · 2025-03-20T18:38:33Z

proposals/infra/INF-0005-Long-Vector-TestPlan.md

+- vector<TYPE, 16> : This size of 'vector' previously only appeared as matrices.
+- vector<TYPE, 17> : Larger than any vector previously possible.
+- vector<TYPE, 1024> : The new max size of a vector. Test for float, half,
+  double, and int64.


" Test for float, half, double, and int64.'" this is for all the above sizes right? What about packed types?

WIP Long vector test plan

2f45d66

github-actions bot added the needs-triage label Mar 7, 2025

damyanp reviewed Mar 7, 2025

View reviewed changes

alsepkow added 2 commits March 10, 2025 18:10

Minor cleanup and format to 80 columns

91b7bc3

Some updates

53261a3

alsepkow marked this pull request as ready for review March 12, 2025 17:49

damyanp reviewed Mar 13, 2025

View reviewed changes

alsepkow added 2 commits March 13, 2025 17:10

Some review and document formatting fixes

9c36dc1

Minor formatting issue

b0028b5

damyanp reviewed Mar 14, 2025

View reviewed changes

proposals/infra/INF-0005-Long-Vector-TestPlan.md Show resolved Hide resolved

damyanp reviewed Mar 14, 2025

View reviewed changes

proposals/infra/INF-0005-Long-Vector-TestPlan.md Outdated Show resolved Hide resolved

alsepkow added 2 commits March 13, 2025 17:33

Fix link

ecb3db9

Fix link

6d6a57c

pow2clk reviewed Mar 17, 2025

View reviewed changes

alsepkow added 3 commits March 18, 2025 21:17

Some updates from PR review

d63eec9

fix line wrap in table

8d3cd35

Remove casts, update intrinsics table with llvm ops, and formatting.

4385044

damyanp reviewed Mar 20, 2025

View reviewed changes

anupamachandra reviewed Mar 20, 2025

View reviewed changes

proposals/infra/INF-0005-Long-Vector-TestPlan.md Show resolved Hide resolved

anupamachandra reviewed Mar 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Long vector test plan #421

Long vector test plan #421

alsepkow commented Mar 7, 2025 •

edited

Loading

damyanp left a comment

damyanp Mar 7, 2025

alsepkow Mar 10, 2025

damyanp Mar 13, 2025

alsepkow Mar 13, 2025

alsepkow commented Mar 10, 2025

damyanp Mar 13, 2025

pow2clk Mar 17, 2025

alsepkow Mar 19, 2025

damyanp Mar 13, 2025

alsepkow Mar 13, 2025

damyanp Mar 13, 2025

damyanp Mar 13, 2025

pow2clk left a comment

pow2clk Mar 17, 2025

alsepkow Mar 19, 2025

pow2clk Mar 17, 2025

pow2clk Mar 17, 2025

alsepkow Mar 19, 2025

damyanp left a comment

damyanp Mar 20, 2025

damyanp Mar 20, 2025

anupamachandra Mar 20, 2025

alsepkow Mar 20, 2025

damyanp Mar 20, 2025

alsepkow Mar 20, 2025 •

edited

Loading

alsepkow Mar 21, 2025

anupamachandra Mar 20, 2025


		# Vector Sizes to test

		I don't think there are any particularly interesting vector sizes to test. So I

		vectors across buffer types. __TODO: Add details about which types of
		buffers to test and why.__

		\| atan2 \| Atan \| CreateFDiv, CreateFAdd, CreateFSub, CreateFCmpOLT,
		\|\|\| CreateFCpmOEQ, CreateFCmpOGE, CreateFCmpOLT, CreateAnd, CreateSelect \| \|

Long vector test plan #421

Are you sure you want to change the base?

Long vector test plan #421

Conversation

alsepkow commented Mar 7, 2025 • edited Loading

damyanp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alsepkow commented Mar 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pow2clk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damyanp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alsepkow Mar 20, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alsepkow commented Mar 7, 2025 •

edited

Loading

alsepkow Mar 20, 2025 •

edited

Loading