Malaysia Open Data MCP

MCP Endpoint: https://mcp.techmavie.digital/datagovmy/mcp

Analytics Dashboard: https://mcp.techmavie.digital/datagovmy/analytics/dashboard

MCP (Model Context Protocol) server for Malaysia's Open Data APIs, providing easy access to government datasets and collections.

Do note that this is NOT an official MCP server by the Government of Malaysia or anyone from Malaysia's Open Data/Jabatan Digital Negara/Ministry of Digital team.

# Features

• Enhanced Unified Search with flexible tokenization and synonym expansion
  • Intelligent query handling with term normalization
  • Support for plurals and common prefixes (e.g., "e" in "epayment")
  • Smart prioritization for different data types
• Parquet File Support using pure JavaScript
  • Parse Parquet files directly in the browser or Node.js
  • Support for BROTLI compression
  • Intelligent date field handling for empty date objects
  • Increased row limits (up to 500 rows) for comprehensive data retrieval
  • Fallback to metadata estimation when parsing fails
  • Automatic dashboard URL mapping for visualization
• Hybrid Data Access Architecture
  • Pre-generated static indexes for efficient searching
  • Dynamic API calls for detailed metadata
• Multi-Provider Geocoding
  • Support for Google Maps, GrabMaps, and Nominatim (OpenStreetMap)
  • Intelligent service selection based on location and available API keys
  • GrabMaps optimization for locations in Malaysia
  • Automatic fallback between providers
• Comprehensive Data Sources
  • Malaysia's Data Catalogue with rich metadata
  • Interactive Dashboards for data visualization
  • Department of Statistics Malaysia (DOSM) data
  • Weather forecast and warnings
  • Public transport and GTFS data
• Multi-Provider Malaysian Geocoding
  • Optimized for Malaysian addresses and locations
  • Three-tier geocoding system: GrabMaps, Google Maps, and Nominatim
  • Prioritizes local knowledge with GrabMaps for better Malaysian coverage
  • Automatic fallback to Nominatim when no API keys are provided

# Architecture

This MCP server implements a hybrid approach for efficient data access:

• Pre-generated Static Indexes for listing and searching datasets and dashboards
• Dynamic API Calls only when specific dataset or dashboard details are requested

This approach provides several benefits:

• Faster search and listing operations
• Reduced API calls to external services
• Consistent data access patterns
• Up-to-date detailed information when needed

# Documentation

• TOOLS.md - Detailed information about available tools and best practices
• PROMPT.md - AI integration guidelines and usage patterns

# AI Integration

When integrating this MCP server with AI models:

1. Use the unified search tool first - Always start with search_all for any data queries
2. Follow the correct URL patterns - Use https://data.gov.my/... and https://open.dosm.gov.my/...
3. Leverage Parquet file tools - Use parse_parquet_file to access data directly or get_parquet_info for metadata
4. Use the hybrid approach - Static indexes for listing/searching, API calls for details
5. Consider dashboard visualization - For complex data, use the dashboard links provided by find_dashboard_for_parquet
6. Leverage the multi-provider Malaysian geocoding - For Malaysian location queries, the system automatically selects the best provider (GrabMaps, Google Maps, or Nominatim) with fallback to Nominatim when no API keys are configured

Refer to PROMPT.md for comprehensive AI integration guidelines.

# Installation

npm install

# Quick Start (Hosted Server)

The easiest way to use this MCP server is via the hosted endpoint. No installation required!

Server URL:

https://mcp.techmavie.digital/datagovmy/mcp

# # # Using Your Own API Keys

You can provide your own API keys via URL query parameters:

https://mcp.techmavie.digital/datagovmy/mcp?googleMapsApiKey=YOUR_KEY

Or via headers:

• X-Google-Maps-Api-Key: YOUR_KEY
• X-GrabMaps-Api-Key: YOUR_KEY
• X-AWS-Access-Key-Id: YOUR_KEY
• X-AWS-Secret-Access-Key: YOUR_KEY
• X-AWS-Region: ap-southeast-5

Supported Query Parameters:

| Parameter | Description |
|- -- -- -- -- --|- -- -- -- -- -- --|
| googleMapsApiKey | Google Maps API key for geocoding |
| grabMapsApiKey | GrabMaps API key for Southeast Asia geocoding |
| awsAccessKeyId | AWS Access Key ID for AWS Location Service |
| awsSecretAccessKey | AWS Secret Access Key |
| awsRegion | AWS Region (default: ap-southeast-5) |

⚠️ Important: GrabMaps Requirements

To use GrabMaps geocoding, you need ALL FOUR parameters:

• grabMapsApiKey
• awsAccessKeyId
• awsSecretAccessKey
• awsRegion

GrabMaps uses AWS Location Service under the hood, so AWS credentials are required alongside the GrabMaps API key.

# # Client Configuration

For Claude Desktop / Cursor / Windsurf, add to your MCP configuration:

{
  "mcpServers": {
    "malaysia-opendata": {
      "transport": "streamable-http",
      "url": "https://mcp.techmavie.digital/datagovmy/mcp"
    }
  }
}

With your own API key:

{
  "mcpServers": {
    "malaysia-opendata": {
      "transport": "streamable-http",
      "url": "https://mcp.techmavie.digital/datagovmy/mcp?googleMapsApiKey=YOUR_KEY"
    }
  }
}

# Self-Hosted (VPS)

If you prefer to run your own instance, see deploy/DEPLOYMENT.md for detailed VPS deployment instructions with Docker and Nginx.

# Analytics Dashboard

The hosted server includes a built-in analytics dashboard:

Dashboard URL: https://mcp.techmavie.digital/datagovmy/analytics/dashboard

# # Analytics Endpoints

| Endpoint | Description |
|- -- -- -- -- -|- -- -- -- -- -- --|
| /analytics | Full analytics summary (JSON) |
| /analytics/tools | Detailed tool usage stats (JSON) |
| /analytics/dashboard | Visual dashboard with charts (HTML) |

The dashboard tracks:

• Total requests and tool calls
• Tool usage distribution
• Hourly request trends (last 24 hours)
• Requests by endpoint
• Top clients by user agent
• Recent tool calls feed

Auto-refreshes every 30 seconds.

# Available Tools

# # Data Catalogue

• list_datasets: Lists available datasets in the Data Catalogue
• get_dataset: Gets data from a specific dataset in the Data Catalogue
• search_datasets: Searches for datasets in the Data Catalogue

# # Department of Statistics Malaysia (DOSM)

• list_dosm_datasets: Lists available datasets from DOSM
• get_dosm_dataset: Gets data from a specific DOSM dataset

# # Parquet File Handling

• parse_parquet_file: Parse and display data from a Parquet file URL
  • Supports up to 500 rows for comprehensive data analysis
  • Automatically handles empty date objects with appropriate formatting
  • Processes BigInt values for proper JSON serialization
• get_parquet_info: Get metadata and structure information about a Parquet file
• find_dashboard_for_parquet: Find the corresponding dashboard URL for a Parquet file

# # Weather

• get_weather_forecast: Gets weather forecast for Malaysia
• get_weather_warnings: Gets current weather warnings for Malaysia
• get_earthquake_warnings: Gets earthquake warnings for Malaysia

# # Transport

• list_transport_agencies: Lists available transport agencies with GTFS data
• get_transport_data: Gets GTFS data for a specific transport agency

# # GTFS Parsing

• parse_gtfs_static: Parses GTFS Static data (ZIP files with CSV data) for a specific transport provider
• parse_gtfs_realtime: Parses GTFS Realtime data (Protocol Buffer format) for vehicle positions
• get_transit_routes: Extracts route information from GTFS data
• get_transit_stops: Extracts stop information from GTFS data, optionally filtered by route

# # Test

• hello: A simple test tool to verify that the MCP server is working correctly

# Data-Catalogue Information Retrieval

The MCP server provides robust handling for data-catalogue information retrieval:

# # Date Handling in Parquet Files

• Empty Date Objects: The system automatically detects and handles empty date objects in parquet files
• Dataset-Specific Handling: Special handling for known datasets like employment_sector with annual data from 2001-2022
• Pattern Recognition: Detects date patterns in existing data to maintain consistent formatting
• Increased Row Limits: Supports up to 500 rows (increased from 100) for more comprehensive data analysis

# # BigInt Processing

• Automatic Serialization: BigInt values are automatically converted to strings for proper JSON serialization
• Type Preservation: Original types are preserved in the schema information

# # Schema Detection

• Automatic Type Inference: Detects column types including special handling for date fields
• Consistent Representation: Ensures date fields are consistently represented as strings

# Usage Examples

# # Get Weather Forecast

const result = await tools.get_weather_forecast({
  location: "Kuala Lumpur",
  days: 3
});

# # Search Datasets

const result = await tools.search_datasets({
  query: "population",
  limit: 5
});

# # Parse GTFS Data

// Parse GTFS Static data
const staticData = await tools.parse_gtfs_static({
  provider: "ktmb"
});

// Get real-time vehicle positions
const realtimeData = await tools.parse_gtfs_realtime({
  provider: "prasarana",
  category: "rapid-rail-kl"
});

// Get transit routes
const routes = await tools.get_transit_routes({
  provider: "mybas-johor"
});

// Get stops for a specific route
const stops = await tools.get_transit_stops({
  provider: "prasarana",
  category: "rapid-rail-kl",
  route_id: "LRT-KJ"
});

# API Rate Limits

Please be aware of rate limits for the underlying APIs. Excessive requests may be throttled.

# Project Structure

• src/index.ts: Main MCP server implementation and tool registration
• src/http-server.ts: Streamable HTTP server for VPS deployment
• src/datacatalogue.tools.ts: Data Catalogue API tools
• src/dashboards.tools.ts: Dashboard access and search tools
• src/dosm.tools.ts: Department of Statistics Malaysia tools
• src/unified-search.tools.ts: Enhanced unified search with tokenization and synonym expansion
• src/parquet.tools.ts: Parquet file parsing and metadata tools
• src/weather.tools.ts: Weather forecast and warnings tools
• src/transport.tools.ts: Transport and GTFS data tools
• src/gtfs.tools.ts: GTFS parsing and analysis tools
• src/flood.tools.ts: Flood warning and monitoring tools
• Dockerfile: Docker configuration for VPS deployment
• docker-compose.yml: Docker Compose configuration
• deploy/: Deployment files (nginx config, deployment guide)
• package.json: Project dependencies and scripts
• tsconfig.json: TypeScript configuration

# Local Development

#  Install dependencies
npm install

#  Run HTTP server in development mode
npm run dev:http

#  Or build and run production version
npm run build
npm run start:http

#  Test health endpoint
curl http://localhost:8080/health

#  Test MCP endpoint
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

# Troubleshooting

# # Container Issues

#  Check container status
docker compose ps

#  View logs
docker compose logs -f

#  Restart container
docker compose restart

# # Test MCP Connection

#  List tools
curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

#  Call hello tool
curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"my_hello","arguments":{}}}'

# Configuration

# # Environment Variables

This project supports the following configuration options:

Geocoding Credentials (Optional. Only for GTFS Transit Features Usage):

The following credentials are only needed if you plan to use the GTFS transit tools that require geocoding services. Other features like data catalogue access, weather forecasts, and DOSM data do not require these credentials.

• googleMapsApiKey: Optional. If provided, the system will use Google Maps API for geocoding location names to coordinates.
• grabMapsApiKey: Optional. Required for GrabMaps geocoding, which is optimized for locations in Malaysia.
• awsAccessKeyId: Required for GrabMaps integration. AWS access key for GrabMaps API authentication.
• awsSecretAccessKey: Required for GrabMaps integration. AWS secret key for GrabMaps API authentication.
• awsRegion: Required for GrabMaps integration. AWS region for GrabMaps API (e.g. 'ap-southeast-5' for Malaysia region or ap-southeast-1 for Singapore region).

If neither Google Maps nor GrabMaps API keys are provided, the GTFS transit tools will automatically fall back to using Nominatim (OpenStreetMap) API for geocoding, which is free and doesn't require credentials.

You can set these configuration options in two ways:

1. Via URL query parameters when connecting to the hosted server (see Quick Start section)
2. As environment variables for local development or self-hosted deployment

# # # Setting up environment variables

Create a .env file in the root directory:

GOOGLE_MAPS_API_KEY=your_google_api_key_here
GRABMAPS_API_KEY=your_grab_api_key_here
AWS_ACCESS_KEY_ID=your_aws_access_key_for_grabmaps
AWS_SECRET_ACCESS_KEY=your_aws_secret_key_for_grabmaps
AWS_REGION=ap-southeast-5

The variables will be automatically loaded when you run the server.

Note: For Malaysian locations, GrabMaps provides the most accurate geocoding results, followed by Google Maps. If you don't provide either API key, the system will automatically use Nominatim API instead, which is free but may have less accurate results for some locations in Malaysia.

Important: These geocoding credentials are only required for the following GTFS transit tools:

• get_transit_routes - When converting location names to coordinates
• get_transit_stops - When converting location names to coordinates
• parse_gtfs_static - When geocoding is needed for stop locations

Note about GTFS Realtime Tools: The parse_gtfs_realtime tool is currently in development and has limited availability. Real-time data access through this MCP is experimental and may not be available for all providers or routes. For up-to-date train and bus schedules, bus locations, and arrivals in real-time, please use official transit apps like Google Maps, MyRapid PULSE, Moovit, or Lugo.

All other tools like data catalogue access, dashboard search, weather forecasts, and DOSM data do not require any geocoding credentials.

# License

MIT - See LICENSE file for details.

# Acknowledgments

• Malaysia Open Data Portal
• Department of Statistics Malaysia
• Malaysian Meteorological Department
• Google Maps Platform for geocoding
• GrabMaps for geocoding
• Nominatim for geocoding
• Model Context Protocol for the MCP framework