Port lexer and parser from the PowerShell repository

[ForNeVeR]

While our recent struggles with the lexer and the parser so far weren't so fruitful (e.g., see #236 and #258), I have an idea of a radical improvement.

What if we try to port the whole lexer and parser from the actual PowerShell repo?

Let's see the following classes:

• System.Management.Automation.Language.Tokenizer
• System.Management.Automation.Language.Parser

And see if it's possible to port them as-is (could be very tricky of course, but we can try!)

This will guarantee we have the last and the most correct definitions of everything in the original repo.

Our current definitions are not bad per se, but they are created from old PowerShell spec and thus are very outdated.

And the new spec, unfortunately, doesn't exist.

[ForNeVeR]

The current plan is:

1. Investigate if the types correspond to what we require. Whether the Tokenizer generates types that are ok for our lexer, and Parser generates types that correspond to IntelliJ's parser type structure.
2. Investigate if the API is ok. E.g. IntelliJ has some resumable lexers and whatnot: can we implement the same on the PowerShell lexer structure? Probably not. But would it be ok to re-lex the whole file each time?
3. Extract the lexer/parser from the main PowerShell repo, isolate from the actual execution engine.
4. Implement a prototype on IPC.
5. Create the corresponding types on the JVM side.1.
6. Extract the isolated lexer/parser to a separate C# process.
7. Run the isolated parser/lexer in IntelliJ, get the results via IPC (JSON over stdout would work, or Rd, or whatever).
8. And now, think about porting the C# code to Kotlin or Java. Prefer some automated converter, but manual conversion that preserves the code structure would also be ok. My purpose is to be able to easily port the further changes back.