Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Exponential growth of template processing time for large repositories due to shared RawModels #10506

Open
banchan86 opened this issue Jan 16, 2025 · 0 comments
Labels
template The stock site template

Comments

@banchan86
Copy link

banchan86 commented Jan 16, 2025

Describe the bug
When using the templating system it is often desirable to set the isShared flag in the function exports.getOptions to true in order to access information from other objects to customize the API template. However, this leads to growing of the RawModel size, which becomes an issue especially for large repositories and leads to exponential growth of the applying template to X models part of the docfx build process.

To Reproduce
Steps to reproduce the behavior:

  1. Choose a large repository (few hundred - thousand API objects)
  2. Create a template folder with a file called 'ManagedReference.overwrite.js`.
exports.getOptions = function (model) {
    return {
        isShared: true
    };
}
  1. Add template folder to docfx.json template property.
  2. Run
dotnet docfx --exportRawModel  
  1. On a repository that produces ~1000 models, this takes 30 minutes on my computer and generates ~1000 individual RawModel files that are each ~70Mb in size and >1.8 million lines. Even if the RawModel files are not being exported, template processing is very slow, so I think the problem is just that the input data model is just very large which affects the exports.transform function for every item.

Expected Behaviour
While I think it is expected behaviour, I think it could be optimized.

Desired Solution
Is there a way to control what is being shared, such that the RawModel size is not so large? I see two possible options:

  1. Filtering which RawModels to share.
  2. Filtering which items in a RawModel are being shared.

This might already be doable with exports.getOptions, but I am not sure how, so if someone can point me to an example that would be great!

Context (please complete the following information):

  • Browser: Chrome
  • Form factor: Desktop
  • Docfx version: 2.78.2
  • Template: default, custom.
@banchan86 banchan86 added bug template The stock site template labels Jan 16, 2025
@banchan86 banchan86 changed the title [Bug] Exponential growth of RawModel size when shared [Bug] Exponential growth of template procressing time for large repositories due to shared RawModels Jan 16, 2025
@banchan86 banchan86 changed the title [Bug] Exponential growth of template procressing time for large repositories due to shared RawModels [Bug] Exponential growth of template processing time for large repositories due to shared RawModels Jan 16, 2025
@yufeih yufeih removed the bug label Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
template The stock site template
Projects
None yet
Development

No branches or pull requests

2 participants