trino/presto和superset通过docker-compose安装连接minio

2022-05-12 15:28:43

1、下载docker-compose(使用命令下载或者手动下载)

wget https://github.com/docker/compose/releases/download/1.28.6/docker-compose-Linux-x86_64

2、创建/修改配置文件

docker-compose.yml （待修改参数：[MINIO_ACCESS_KEY]、[MINIO_SECRET_KEY]）

version: '3.7'

volumes:
  elasticsearch-data:
  redis-data:
  postgres-data:
  minio-data:
  trino-node-data:

networks:
  estack:

services:

  redis:
    image: redis:5
    container_name: superset_cache
    ports:
      - "127.0.0.1:6379:6379"
    volumes:
      - redis-data:/data
    networks:
      - estack
      
  postgres:
    image: postgres:12
    environment:
      POSTGRES_DB: superset
      POSTGRES_PASSWORD: superset
      POSTGRES_USER: superset
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - estack
      
  superset:
    container_name: superset
    image: amancevice/superset:1.0.1
    depends_on:
      - postgres
      - redis
    environment:
      MAPBOX_API_KEY: ${MAPBOX_API_KEY}
    ports:
      - "8088:8088"
    volumes:
      - /etc/localtime:/etc/localtime
      - ./conf/superset/superset_config.py:/etc/superset/superset_config.py
    networks:
      - estack
  trino:
    image: trinodb/trino:353
    volumes:
      - minio-data:/data
    ports:
      - "9080:8080"
    volumes:
      - ./conf/trino/catalog/datalake.properties:/etc/trino/catalog/datalake.properties:ro
      - trino-node-data:/data/trino
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    restart: on-failure

trino/etc/jvm.config

-server
-Xmx1G
-XX:-UseBiasedLocking
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+ExplicitGCInvokesConcurrent
-XX:+ExitOnOutOfMemoryError
-XX:+HeapDumpOnOutOfMemoryError
-XX:ReservedCodeCacheSize=512M
-XX:PerMethodRecompilationCutoff=10000
-XX:PerBytecodeRecompilationCutoff=10000
-Djdk.attach.allowAttachSelf=true
-Djdk.nio.maxCachedBufferSize=2000000

trino/etc/node.properties

node.environment=development
node.id=0bc6c957-0404-490f-8f6a-8db6e8f3cf2b
node. style="margin: 1.4em 0px; padding-left: 1em; color: rgb(100, 100, 100); border-left: 3px solid rgb(211, 211, 211); font-family: -apple-system, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Microsoft YaHei", "Source Han Sans SC", "Noto Sans CJK SC", "WenQuanYi Micro Hei", sans-serif; font-size: medium; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial;">trino/catalog/datalake.properties （待修改参数：[hive.s3.endpoint]、[hive.s3.aws-access-key]、[hive.s3.aws-secret-key]）connector.name=hive-hadoop2
hive.metastore=file
hive.s3-file-system-type=TRINO
hive.metastore.catalog.dir=s3://datalake/catalog
hive.s3.aws-access-key=minio
hive.s3.aws-secret-key=minio123
hive.s3.endpoint=minio:9000      # minio_ip:port
hive.s3.path-style-access=true
hive.s3select-pushdown.enabled=true
hive.s3.ssl.enabled=false
superset/superset_config.py
import os

BABEL_DEFAULT_LOCALE = "zh"
MAPBOX_API_KEY = os.getenv('MAPBOX_API_KEY', '')
CACHE_CONFIG = {
    'CACHE_TYPE': 'redis',
    'CACHE_DEFAULT_TIMEOUT': 300,
    'CACHE_KEY_PREFIX': 'superset_',
    'CACHE_REDIS_HOST': 'redis',
    'CACHE_REDIS_PORT': 6379,
    'CACHE_REDIS_DB': 1,
    'CACHE_REDIS_URL': 'redis://redis:6379/1'
}
SQLALCHEMY_DATABASE_URI = \
    'postgresql+psycopg2://superset:superset@postgres:5432/superset'
SQLALCHEMY_TRACK_MODIFICATIONS = True
SECRET_KEY = 'thisISaSECRET_1234'
3、启动容器
docker-compose up -d
完成！
原文链接：https://zhuanlan.zhihu.com/p/361302542

分享好友

分享这个小栈给你的朋友们，一起进步吧。

Trino

创建时间：2022-04-12 14:37:38

Trino

展开

订阅须知

• 所有用户可根据关注领域订阅专区或所有专区

• 付费订阅：虚拟交易，一经交易不退款；若特殊情况，可3日内客服咨询

• 专区发布评论属默认订阅所评论专区（除付费小栈外）

技术专家

查看更多

飘絮絮絮丶
专家