Skip to content

Issues: confident-ai/deepeval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

143 Open 228 Closed
143 Open 228 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Synthesizer is not able to take default model as a critic model
#1468 opened Mar 26, 2025 by spike-spiegel-21
1
TestResult misses additional_metadata
#1466 opened Mar 25, 2025 by jakob-puehringer-tv
Error when using generate_goldens_from_scratch
#1465 opened Mar 24, 2025 by NaNa7Miiii
2
Error when running the documentation exemple for DAG eval
#1457 opened Mar 20, 2025 by Bennoo
5
Provide ability to disable tqdm
#1454 opened Mar 19, 2025 by sergeyklay
RedTeamer is broken
#1443 opened Mar 17, 2025 by tonylincon1
6
programmatic alternative to KEY_FILE_HANDLER.write_key
#1439 opened Mar 16, 2025 by codefromthecrypt
Using OpenAI on Azure fails with TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str' exception
#1438 opened Mar 16, 2025 by moshevi
Synthesizer ChromaDB - no required index generated
#1430 opened Mar 13, 2025 by csanadpoda
8
Schema Error when using Vertexai Custom LLM Evaluation
#1421 opened Mar 6, 2025 by huiyi-liang
4
Custom LLM in the docs provide no schema input for generate method, and returns string, which causes summarization.py raise AttributeError: 'str' object has no attribute 'truths'
#1420 opened Mar 6, 2025 by pin-lpt-heatherwick
4
pytest plugin 'plugins' has generic name, triggers telemetry by default.
#1419 opened Mar 6, 2025 by Ben-grmbl
1
False login when using login_with_confident_api_key with no key
#1418 opened Mar 6, 2025 by AGTGreg
Don't push for confidently.ai platform, or give way to silence it
#1417 opened Mar 6, 2025 by bcmcmahan
Write to local directory when write_cache = False
#1413 opened Mar 4, 2025 by EZHU331
Confusing tools in the Tool Correctness metric reason
#1410 opened Mar 4, 2025 by KamilFatkhiev
missing 1 required positional argument: 'schema' for DAGMetric when using custom LLM
#1390 opened Feb 26, 2025 by mohamed-ahshik
1
Request timeout when using multiple metrics
#1376 opened Feb 21, 2025 by vinhqdang
2
Verbose printing goes into infinite loop in notebook help wanted Extra attention is needed
#1363 opened Feb 14, 2025 by penguine-ip
1
o3-mini is not working with G-Eval due to wrong temperature and logprobs parameter
#1358 opened Feb 12, 2025 by RicardoBochnia
1
Feature Request: Return both score and reason from metric.measure()
#1350 opened Feb 8, 2025 by Mujae
2
Change to a version of gRPC >= 1.70.0 as the current verison (1.67.1) has a known vulnerability.
#1343 opened Feb 4, 2025 by harshapraneeth
query about trying to add test cases from csv file
#1333 opened Feb 2, 2025 by Sara-Hossny
5
OpenAI Batch API
#1286 opened Jan 19, 2025 by luarss
1
Benchmarks introduction does setup wrong
#1284 opened Jan 18, 2025 by AMindToThink
3
ProTip! What’s not been updated in a month: updated:<2025-02-26.