Skip to content

Commit c777f96

Browse files
authored
Merge branch 'main' into task/gsk-2701-add-documentation-advanced-scan
2 parents 984ced2 + a093292 commit c777f96

File tree

119 files changed

+9579
-1874
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

119 files changed

+9579
-1874
lines changed

.github/workflows/build-python.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ jobs:
7272
pandas_v1: false
7373
langchain_minimal: false
7474
- python-version: "3.10"
75-
os: macos-latest
75+
os: macos-12
7676
pydantic_v1: false
7777
pandas_v1: false
7878
langchain_minimal: false
@@ -144,7 +144,7 @@ jobs:
144144
df -h
145145
146146
- name: Re-install lightgbm from sources for MacOS
147-
if: ${{ matrix.os == 'macos-latest' }}
147+
if: ${{ matrix.os == 'macos-12' }}
148148
run: |
149149
pdm run pip uninstall lightgbm -y
150150
pdm run pip install --no-binary lightgbm lightgbm --config-settings=cmake.define.USE_OPENMP=OFF

.github/workflows/create-release.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ jobs:
6060

6161
- name: "@slack Release process started"
6262
id: slack
63-
uses: slackapi/slack-github-action@v1.25.0
63+
uses: slackapi/slack-github-action@v1.26.0
6464
with:
6565
channel-id: ${{ vars.SLACK_CHANNEL_ID }}
6666
slack-message: |-

.github/workflows/post-release.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ jobs:
117117
- name: "@slack Share release process completion"
118118
# cancellable always() https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions
119119
if: ${{ !cancelled() }}
120-
uses: slackapi/slack-github-action@v1.25.0
120+
uses: slackapi/slack-github-action@v1.26.0
121121
env:
122122
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
123123
ON_SUCCESS: |-

CONTRIBUTING.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,9 @@ An awesome feature request addresses the following points:
4848

4949
## Style guide
5050

51-
We use `black` to format the code and `isort` to sort the imports. You can automatically enforce this style using the pre-commit hooks. You can find the instructions to install the pre-commit hooks [here](https://docs.giskard.ai/en/latest/community/contribution_guidelines/dev-environment.html).
51+
We use `black` to format the code and `isort` to sort the imports. You can automatically enforce this style using the
52+
pre-commit hooks. You can find the instructions to install the pre-commit
53+
hooks [here](https://docs.giskard.ai/en/stable/community/contribution_guidelines/dev-environment.html).
5254

5355
For documentation strings, Giskard follows the [numpydoc](https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard) style.
5456

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
</div>
1818
<h3 align="center">
19-
<a href="https://docs.giskard.ai/en/latest/getting_started/index.html"><b>Docs</b></a> &bull;
19+
<a href="https://docs.giskard.ai/en/stable/getting_started/index.html"><b>Docs</b></a> &bull;
2020
<a href="https://www.giskard.ai/knowledge-categories/news/?utm_source=github&utm_medium=github&utm_campaign=github_readme&utm_id=readmeblog"><b>Blog</b></a> &bull;
2121
<a href="https://www.giskard.ai/?utm_source=github&utm_medium=github&utm_campaign=github_readme&utm_id=readmeblog"><b>Website</b></a> &bull;
2222
<a href="https://gisk.ar/discord"><b>Discord</b></a>

docs/assets/scan_vision.png

81.3 KB
Loading

docs/assets/scan_widget.html

+5-5
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@
131131
&lt;li&gt;Drift between your training set and test set&lt;/li&gt;
132132
&lt;/ul&gt;
133133
&lt;p class=&quot;mt-1&quot;&gt;
134-
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/performance_bias/index.html&quot; target=&quot;_blank&quot;&gt;guide on performance bias.&lt;/a&gt;
134+
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/performance_bias/index.html&quot; target=&quot;_blank&quot;&gt;guide on performance bias.&lt;/a&gt;
135135
&lt;/p&gt;
136136
137137
&lt;/div&gt;
@@ -1129,7 +1129,7 @@
11291129
&lt;li&gt;Use of complex models with large number of parameters that tend to overfit the training data&lt;/li&gt;
11301130
&lt;/ul&gt;
11311131
&lt;p class=&quot;mt-1&quot;&gt;
1132-
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/robustness/index.html&quot; target=&quot;_blank&quot;&gt;guide on robustness issues.&lt;/a&gt;
1132+
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/robustness/index.html&quot; target=&quot;_blank&quot;&gt;guide on robustness issues.&lt;/a&gt;
11331133
&lt;/p&gt;
11341134
11351135
&lt;/div&gt;
@@ -1257,7 +1257,7 @@
12571257
class&lt;/li&gt;
12581258
&lt;/ul&gt;
12591259
&lt;p class=&quot;mt-1&quot;&gt;
1260-
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/overconfidence/index.html&quot; target=&quot;_blank&quot;&gt;guide on overconfidence issues.&lt;/a&gt;
1260+
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/overconfidence/index.html&quot; target=&quot;_blank&quot;&gt;guide on overconfidence issues.&lt;/a&gt;
12611261
&lt;/p&gt;
12621262
12631263
&lt;/div&gt;
@@ -2233,7 +2233,7 @@
22332233
&lt;li&gt;Data noise: the training set contains anomalies that are unrelated to the underlying problem (data collection, measurement biases, data preprocessing issues, etc.)&lt;/li&gt;
22342234
&lt;/ul&gt;
22352235
&lt;p class=&quot;mt-1&quot;&gt;
2236-
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/spurious/index.html&quot; target=&quot;_blank&quot;&gt;guide on spurious correlation.&lt;/a&gt;
2236+
To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/knowledge/key_vulnerabilities/spurious/index.html&quot; target=&quot;_blank&quot;&gt;guide on spurious correlation.&lt;/a&gt;
22372237
&lt;/p&gt;
22382238
22392239
&lt;/div&gt;
@@ -3026,4 +3026,4 @@
30263026
}), z = {}, window.jQuery !== d && q(window.jQuery), "function" == typeof define && define.amd ? define([], n) : "object" == (typeof module === "undefined" ? "undefined" : _typeof(module)) && "object" == _typeof(module.exports) && (module.exports = n()), window.iFrameResize = window.iFrameResize || n());
30273027
}();
30283028
(function(){iFrameResize({ checkOrigin: false }, '#scan-140417288606800');})();
3029-
</script>
3029+
</script>

docs/conf.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434

3535
autoclass_content = "both"
3636

37-
# autodoc_mock_imports = ["giskard.ml_worker.generated"]
37+
autodoc_mock_imports = ["ragas", "langchain_core"]
3838
templates_path = ["_templates"]
3939
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
4040

docs/getting_started/index.md

+1-5
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Testing Machine Learning applications can be tedious. Since AI models depend on
2424

2525
To help you solve these challenges, Giskard library helps to:
2626

27-
- **Scan your model to find hidden vulnerabilitie automatically**: The `giskard` scan automatically detects vulnerabilities
27+
- **Scan your model to find hidden vulnerabilities automatically**: The `giskard` scan automatically detects vulnerabilities
2828
such as performance bias, hallucination, prompt injection, data leakage, spurious correlation, overconfidence, etc.
2929
<br><br>
3030
<iframe src="https://htmlpreview.github.io/?https://gist.githubusercontent.com/AbSsEnT/a67354621807f3c3a332fca7d8b9a5c8/raw/588f027dc6b14c88c7393c50ff3086fe1122e2e9/LLM_QA_IPCC_scan_report.html" width="700" height="400"></iframe>
@@ -62,7 +62,3 @@ An Enterprise Hub for teams to collaborate on top of the open-source Giskard lib
6262
- 💬 **Collect business feedback** and **share your model results** with data scientists, QA teams and auditors.
6363
<br><br>
6464
![](../assets/credit_scoring_comment.png)
65-
66-
67-
Get started **now** with our [demo HuggingFace Space](https://huggingface.co/spaces/giskardai/giskard) or
68-
by following [installation instructions](../getting_started/quickstart/index.md)! 🐢

docs/getting_started/quickstart/index.md

+8-4
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,17 @@
88
quickstart_llm
99
quickstart_tabular
1010
quickstart_nlp
11+
quickstart_vision
1112
```
1213

13-
1414
:::::{grid} 1 1 2 2
1515

16-
17-
::::{grid-item-card} <br/><h3>📚 LLM Quickstart</h3>
16+
::::{grid-item-card} <br/><h3>📚 LLM Quickstart</h3>
1817
:text-align: center
1918
:link: quickstart_llm.ipynb
2019
::::
2120

22-
::::{grid-item-card} <br/><h3>📊 Tabular Quickstart</h3>
21+
::::{grid-item-card} <br/><h3>📊 Tabular Quickstart</h3>
2322
:text-align: center
2423
:link: quickstart_tabular.ipynb
2524
::::
@@ -29,4 +28,9 @@ quickstart_nlp
2928
:link: quickstart_nlp.ipynb
3029
::::
3130

31+
::::{grid-item-card} <br/><h3>📸 Vision Quickstart</h3>
32+
:text-align: center
33+
:link: quickstart_vision.ipynb
34+
::::
35+
3236
:::::

docs/getting_started/quickstart/quickstart_llm.ipynb

+2-2
Original file line numberDiff line numberDiff line change
@@ -1191,7 +1191,7 @@
11911191
"cell_type": "markdown",
11921192
"metadata": {},
11931193
"source": [
1194-
"If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/latest/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
1194+
"If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/stable/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
11951195
]
11961196
},
11971197
{
@@ -1478,7 +1478,7 @@
14781478
"&lt;li&gt;Overreliance on spurious correlations like the presence of specific word&lt;/li&gt;\n",
14791479
"&lt;li&gt;Use of complex models with large number of parameters that tend to overfit the training data&lt;/li&gt;\n",
14801480
"&lt;/ul&gt;\n",
1481-
"&lt;p&gt;To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/robustness/index.html&quot;&gt;guide on robustness issues&lt;/a&gt;.&lt;/p&gt;\n",
1481+
"&lt;p&gt;To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/robustness/index.html&quot;&gt;guide on robustness issues&lt;/a&gt;.&lt;/p&gt;\n",
14821482
" &lt;/div&gt;\n",
14831483
" &lt;/div&gt;\n",
14841484
" &lt;/div&gt;\n",

docs/getting_started/quickstart/quickstart_nlp.ipynb

+7-7
Original file line numberDiff line numberDiff line change
@@ -177,7 +177,7 @@
177177
"id": "3PGiyUwAJpdK"
178178
},
179179
"source": [
180-
"To prepare for the vulnerability scan, make sure to wrap your dataset using Giskard's Dataset class. More details [here](https://docs.giskard.ai/en/latest/open_source/scan/scan_nlp/index.html#step-1-wrap-your-dataset)."
180+
"To prepare for the vulnerability scan, make sure to wrap your dataset using Giskard's Dataset class. More details [here](https://docs.giskard.ai/en/stable/open_source/scan/scan_nlp/index.html#step-1-wrap-your-dataset)."
181181
]
182182
},
183183
{
@@ -258,7 +258,7 @@
258258
"id": "NwunAyRvJzOS"
259259
},
260260
"source": [
261-
"To prepare for the vulnerability scan, make sure to wrap your model using Giskard's Model class. You can choose to either wrap the prediction function (preferred option) or the model object. More details [here](https://docs.giskard.ai/en/latest/open_source/scan/scan_nlp/index.html#step-2-wrap-your-model)."
261+
"To prepare for the vulnerability scan, make sure to wrap your model using Giskard's Model class. You can choose to either wrap the prediction function (preferred option) or the model object. More details [here](https://docs.giskard.ai/en/stable/open_source/scan/scan_nlp/index.html#step-2-wrap-your-model)."
262262
]
263263
},
264264
{
@@ -310,7 +310,7 @@
310310
"source": [
311311
"### Scan your model for vulnerabilities with Giskard\n",
312312
"\n",
313-
"Giskard's scan allows you to detect vulnerabilities in your model automatically. These include performance biases, unrobustness, data leakage, stochasticity, underconfidence, ethical issues, and more. For detailed information about the scan feature, please refer to our [scan documentation](https://docs.giskard.ai/en/latest/open_source/scan/scan_nlp/index.html)."
313+
"Giskard's scan allows you to detect vulnerabilities in your model automatically. These include performance biases, unrobustness, data leakage, stochasticity, underconfidence, ethical issues, and more. For detailed information about the scan feature, please refer to our [scan documentation](https://docs.giskard.ai/en/stable/open_source/scan/scan_nlp/index.html)."
314314
]
315315
},
316316
{
@@ -329,7 +329,7 @@
329329
"cell_type": "markdown",
330330
"metadata": {},
331331
"source": [
332-
"If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/latest/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
332+
"If you are running in a notebook, you can display the scan report directly in the notebook using `display(...)`, otherwise you can export the report to an HTML file. Check the [API Reference](https://docs.giskard.ai/en/stable/reference/scan/report.html#giskard.scanner.report.ScanReport) for more details on the export methods available on the `ScanReport` class."
333333
],
334334
"id": "9dd5baaaa6a7ee62"
335335
},
@@ -442,7 +442,7 @@
442442
"&lt;li&gt;Overreliance on spurious correlations like the presence of specific word&lt;/li&gt;\n",
443443
"&lt;li&gt;Use of complex models with large number of parameters that tend to overfit the training data&lt;/li&gt;\n",
444444
"&lt;/ul&gt;\n",
445-
"&lt;p&gt;To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/robustness/index.html&quot;&gt;guide on robustness issues&lt;/a&gt;.&lt;/p&gt;\n",
445+
"&lt;p&gt;To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/robustness/index.html&quot;&gt;guide on robustness issues&lt;/a&gt;.&lt;/p&gt;\n",
446446
" &lt;/div&gt;\n",
447447
" &lt;/div&gt;\n",
448448
" &lt;/div&gt;\n",
@@ -1021,7 +1021,7 @@
10211021
"&lt;li&gt;Data is reflecting some structural biases and societal prejudices&lt;/li&gt;\n",
10221022
"&lt;li&gt;Use of complex models with large number of parameters that tend to overfit the training data&lt;/li&gt;\n",
10231023
"&lt;/ul&gt;\n",
1024-
"&lt;p&gt;To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/latest/getting-started/key_vulnerabilities/ethics/index.html&quot;&gt;guide on unethical behaviour.&lt;/a&gt;&lt;/p&gt;\n",
1024+
"&lt;p&gt;To learn more about causes and solutions, check our &lt;a href=&quot;https://docs.giskard.ai/en/stable/getting-started/key_vulnerabilities/ethics/index.html&quot;&gt;guide on unethical behaviour.&lt;/a&gt;&lt;/p&gt;\n",
10251025
" &lt;/div&gt;\n",
10261026
" &lt;/div&gt;\n",
10271027
" &lt;/div&gt;\n",
@@ -1835,7 +1835,7 @@
18351835
"* Slicing functions such as detectors of toxicity, hate, emotion, etc\n",
18361836
"* Transformation functions such as generators of typos, paraphrase, style tune, etc\n",
18371837
"\n",
1838-
"To create custom tests, refer to [this page](https://docs.giskard.ai/en/latest/open_source/customize_tests/test_model/index.html).\n",
1838+
"To create custom tests, refer to [this page](https://docs.giskard.ai/en/stable/open_source/customize_tests/test_model/index.html).\n",
18391839
"\n",
18401840
"For demo purposes, we will load a simple unit test (test_f1) that checks if the test F1 score is above the given threshold. For more examples of tests and functions, refer to the Giskard catalog."
18411841
]

0 commit comments

Comments
 (0)