feat: Migrate MCP server from fastapi_mcp to MCP Python SDK

[patelchaitany]

Closes #6252

fastapi_mcp has no cycle detection in resolve_schema_references(). Any self-referencing Pydantic model in the OpenAPI schema (compound filters, nested FilterExpression, etc.) causes a RecursionError that takes down the whole MCP server. It also spits out $ref/$defs-heavy schemas that LLMs can't work with, and generates tool names from HTTP verbs (post_get_online_features).

This swaps fastapi_mcp for the MCP Python SDK (FastMCP). Tools are registered with @mcp.tool() and call FeatureStore directly, no internal HTTP round-trip.

What changed

• mcp_server.py -- replaced FastApiMCP wrapper with a create_mcp_server() function that registers tools explicitly
• New registry tools: list_feature_views, list_entities, list_feature_services, list_data_sources
• New data access tools: get_online_features, retrieve_online_documents
• New write tools: write_to_online_store
• New materialization tools: materialize, materialize_incremental
• SSE and streamable HTTP transports via mcp_transport config
• Configurable mcp_base_path (default /mcp)
• Session manager lifecycle tied to FastAPI lifespan
• Elasticsearch timestamp parsing now handles microseconds
• Unit and integration tests rewritten for the new SDK

Tool comparison: Previous vs New

The old fastapi_mcp approach auto-generated MCP tools from FastAPI REST routes. The new approach uses explicit @mcp.tool() handlers with flat, LLM-friendly schemas and direct FeatureStore calls.

Registry introspection (NEW)

Tool	Previous (fastapi_mcp)	New (MCP Python SDK)
list_feature_views	Not available	Added — lists all feature views with name, entities, features, and tags
list_entities	Not available	Added — lists all entities with name, join key, value type, and tags
list_feature_services	Not available	Added — lists all feature services with name, feature views, and tags
list_data_sources	Not available	Added — lists all data sources with name, type, and tags

Data access

Tool	Previous (fastapi_mcp)	New (MCP Python SDK)
get_online_features	Auto-generated as post_get_online_features from POST /get-online-features with $ref-heavy schema	Rewritten — explicit tool with flat schema, permission checks, direct FeatureStore call
retrieve_online_documents	Auto-generated as post_retrieve_online_documents from POST /retrieve-online-documents	Rewritten — supports vector, text, and hybrid search with api_version parameter

Write

Tool	Previous (fastapi_mcp)	New (MCP Python SDK)
write_to_online_store	Auto-generated as post_write_to_online_store from POST /write-to-online-store	Rewritten — explicit tool with permission checks and transform_on_write support

Materialization

Tool	Previous (fastapi_mcp)	New (MCP Python SDK)
materialize	Auto-generated as post_materialize from POST /materialize	Rewritten — explicit tool with ISO-8601 timestamp parsing and permission checks
materialize_incremental	Auto-generated as post_materialize_incremental from POST /materialize-incremental	Rewritten — explicit tool with permission checks

Removed (not useful as LLM tools)

Previous Tool	Reason for removal
post_push (from POST /push)	Streaming/pipeline ingestion via PushSource — not a typical LLM agent operation
get_health (from GET /health)	Health check — no value as an LLM-callable tool
post_chat / get_chat (from /chat)	Placeholder/stub endpoints with dummy responses

Key improvements across all tools

Aspect	Previous (fastapi_mcp)	New (MCP Python SDK)
Schema format	$ref/$defs-heavy, causes RecursionError on recursive models	Flat Annotated[..., Field(description=...)] schemas
Tool naming	HTTP verb-based (post_get_online_features)	Clean function names (get_online_features)
Data path	HTTP round-trip through FastAPI routes	Direct FeatureStore method calls
Permissions	Relied on FastAPI dependency injection	Explicit assert_permissions() on every tool
Transport	Delegated to fastapi_mcp mount	Configurable SSE or streamable HTTP via mcp_transport

---