20 Python Libraries for AI engineers

1. Pydantic: https://docs.pydantic.dev/latest/

2. Pydantic Settings:

https://docs.pydantic.dev/latest/concepts/pydantic_settings/#usage

3. Python Dotenv:

https://pypi.org/project/python-dotenv/

4. FastAPI:

https://fastapi.tiangolo.com/

5. Celery :

https://pypi.org/project/celery/       – Task queue management

6. Databases:

a. PostgreSQL: https://pypi.org/project/psycopg2/   – SQL

b. MongoDB: https://pypi.org/project/pymongo/       – NOSQL

7. SQLAlchemy:

https://pypi.org/project/SQLAlchemy/

8. Alembic :

https://alembic.sqlalchemy.org/en/latest/    – Data Migration tool

9. Pandas :

https://pandas.pydata.org/

10. LLM Model Providers

a. OpenAI : https://platform.openai.com/docs/overview

b .Ollama : https://ollama.com/ 

c .Gemini : https://aistudio.google.com/app/prompts/new_chat

d. Anthropic : https://www.anthropic.com/

e. Llama: https://www.llama.com/  & Others: XAI: https://x.ai/ , Groq: https://console.groq.com/playground

11. Instructor :

https://python.useinstructor.com/     – LLM structured output

12. LLM Frameworks:

a. Langchain: https://python.langchain.com/docs/introduction/

b. LlamaIndex:https://docs.llamaindex.ai/en/stable/

13. Vector Databases:

a. PineCone: https://www.pinecone.io/

b. Qdrant:https://qdrant.tech/documentation/cloud/

c. PgvectorScale: https://github.com/timescale/pgvectorscale

d. Elastic Search: https://www.elastic.co/enterprise-search/vector-search

14.  Observability:

a. Langfuse : https://langfuse.com/

b. Langsmith:https://docs.smith.langchain.com/

c. Arize.ai: https://phoenix.arize.com/

15.  DSPy:

https://dspy.ai/

16.  Documents Parsers

a .PyMUPDF : https://pymupdf.readthedocs.io/en/latest/tutorial.html

b. PyPDF2: https://pypdf2.readthedocs.io/en/3.x/

c. Google Doc AI: https://cloud.google.com/document-ai?hl=en

d. Azure Doc Intelligence: https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence

e. IBM Open source Docling: https://ds4sd.github.io/docling/

f. AWS Textract: https://aws.amazon.com/textract/

17.   Jinja:

https://jinja.palletsprojects.com/en/stable/   – Dynamic prompts

18 .   Jina.ai :

https://jina.ai/     – URL content extractor

19 .   Agentic: (only few)

a. LangGraph: https://www.langchain.com/langgraph

b. CrewAI:https://www.crewai.com/

c. Phidata: https://www.phidata.com/

d. Autogen: https://microsoft.github.io/autogen/0.2/blog/2023/12/01/AutoGenStudio/

20.    Streamlit:

https://streamlit.io/

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top