Data Connection Information Parameter Specifications
When using the processing automation tool, "Connection Information" needs to be entered to access databases or data files. The following are detailed explanations of the connection information parameters for different data sources.
SuperMap SDX+ (--providerType=sdx)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is sdx, indicating the use of the SuperMap SDX+ spatial data engine to access data. |
| --dbType, --type, -T, --db-type (Required) |
Engine type, supporting access to various databases and vector/raster files.Examples: UDBX: --dbType=UDBX PostGISdatabase: --dbType=PGGIS Vector file:--dbType=VECTORFILE |
| --server, -S (Required) |
Database engine service address or file path. |
| --dataset, --dt (Required) |
Name of the dataset to read or save. |
| --CHARSET | Character set encoding, used only for the VECTORFILE engine type. |
| --driver | Driver name required for the data source connection. |
| --database, --db | Database name for the data source connection. |
| --alias, -A | Data source alias. |
| --user, -U | Username for database login. |
| --password, --pwd | Password for the database user. |
| --maxConnPoolNum | Maximum number of connections for the data source. |
| --numPartitions, --num-parts, -N | Number of partitions, defaults to 0. |
| --partitionField, --part-field | Field name used for partitioning. Uses the SmID field by default. However, if the database engine (e.g., PGGIS) does not have an SmID field, the user needs to specify the partition field. |
| --idField, --id-field | Field used to read the unique feature ID from the feature object. Must ensure the field values are unique. If empty, the default SmID field value will be used. |
| --blockingWrite | Write data in blocks. Can effectively improve performance for large data, defaults to true. |
Examples:
- Connect to UDBX: --providerType=sdx --dbType=UDBX --server=F:\data\landuse.udbx --dataset=DLTB
- Connect to PostGIS: --providerType=sdx --dbType=PGGIS --server=127.0.0.1 --database=postgis --user=postgres --password=uitest --maxConnPoolNum=10 --dataset=DLTB
- Connect to ArcGIS for Oracle: --providerType=sdx --server=127.0.0.1:3452/xe --user=testdb --password=testdb --alias=test --maxConnPoolNum=1 --dataset=dt --dbType=ARCSDE_ORACLE
- Connect to vector file (ShapeFile): --providerType=sdx --server=E:\data\DLTB.shp --dataset=DLTB --CHARSET=ANSI --dbType=VECTORFILE
JDBC (--providerType=jdbc)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is jdbc, indicating the use of the JDBC provided Java API to access data. |
| --dbType, --type, -T, --db-type (Required) |
Only supports PostGIS and OracleSpatial databases. Distributed read/write can be achieved in cluster mode. |
| --host (Required) |
Database service address. |
| --port (Required) |
Database service port number, defaults to 0. |
| --database, --db | Database name for the data source connection. |
| --dataset, --dt (Required) |
Name of the dataset to read or save. |
| --schema | Database schema to access. |
| --user, -U | Username for database login. |
| --password, --pwd | Password for the database user. |
| --numPartitions, --num-parts, -N | Number of partitions, defaults to 0. |
| --partitionField, --part-field | Field name used for partitioning. Uses the SmID field by default. However, if the database engine (e.g., PGGIS) does not have an SmID field, the user needs to specify the partition field. |
| --predicates | Array of partition predicates. |
Examples:
- Connect to OracleSpatial: --providerType=jdbc --host=127.0.0.1 --port=1521 --schema=testosp --database=orcl --user=testosp --password=testosp --dbtype=oracle --table=SMDTV(Note: The dataset table name needs to be filled in here.)
- Connect to PostGIS: --providerType=jdbc --host=127.0.0.1 --port=5432 --schema=postgres --database=uitest --user=postgres --password=uitest --dbtype=postgis --dataset=DLTB
DSF (--providerType=dsf)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is dsf, indicating the use of the DSF (Distributed Spatial File) engine to access data. |
| --path, -P, --url (Required) |
DSF directory address, supports local directories and HDFS directories. |
| --bounds, -B | Query bounds, mainly used for range queries. Invalid when as-dsf-rddis true. Compared to using the box query in Query, queryBounds utilizes the primary index for an additional filter, resulting in higher performance. |
| -fields, --result-fields | Field names to read, multiple fields separated by commas. |
| --as-dsf-rdd | Whether to read as DSFFeatureRDD, defaults to false. If true, reads as DSFFeatureRDD. When performing spatial relationships or spatial judgments between two datasets, ensure the partition indexes of the two DSFFeatureRDD datasets are consistent. Otherwise, the user can set this to falseto read as FeatureRDD, which will uniformly build partition indexes during spatial operations or judgments between two datasets. |
Examples:
- Local directory (Windows system): --providerType=dsf --path=E:/data/Zoo_pt
- Local directory (Linux system): --providerType=dsf --path=/home/data/Zoo_pt
- HDFS directory: --providerType=dsf --path=hdfs://127.0.0.1:9000/data/DLTB
CSV (--providerType=csv)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is csv, indicating the data source is a CSV file. |
| --path, -P, --url (Required) |
CSV file path or directory address, supports local paths and HDFS paths. |
| -fields, -F | Fields to import, separated by commas. If the CSV has a header, pass the field names and set FirstRowIsFieldto true. If no header, use column indices like col0, col1, etc. |
| --geofields, --geo-fields, -GF | Spatial information fields to import, separated by commas. Pass one column for line or polygon data, two columns for point data. More are invalid. If the CSV has a header, pass the field names and set FirstRowIsFieldto true. If no header, use column indices like col0, col1, etc. Note: Only supports longitude and latitude coordinate data in the WGS84 coordinate system, and the range must be within [-180,180],[-90,90]. |
| --firstRowIsField, --first-as-field, -FAF | Whether the first row is the header, used as field names. |
| --idField, --id-field, -IF | Specifies the ID field name. If not specified, defaults to uuid. |
| --modifyDesFieldTypes, --des-field-type, -DFT | Specifies the data type of fields. If not specified, fields are read as string type by default. Input format: 'fieldname->fieldtype,fieldname->fieldtype'. |
| --timeZoneOffset,--time-Zone-Offset,-TZO | When modifyDesFieldTypesspecifies a field type as date, it defaults to UTC time. This parameter can be used for time zone conversion, format is "+/-hh:mm:ss". E.g., convert to Beijing Time: --timeZoneOffset=+08:00:00, meaning UTC+8 hours. |
Examples:
- Local path (Windows system): --providerType=csv --path=E:/data/data.csv
- Local path (Linux system): --providerType=csv --path=/home/data/data.csv
- HDFS path: --providerType=csv --path=hdfs://127.0.0.1:9000/data/data.csv
ESRI File Geodatabase (--providerType=gdb)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is gdb, indicating the data source is an ESRI File Geodatabase. |
| --path, -P, --url (Required) |
GDB directory address, supports local directories and HDFS directories. |
| --dataset, --table | Name of the dataset to read within the GDB. |
| --minPartitions, --min-part, -M | Minimum number of Spark RDD partitions, defaults to 0. |
Examples:
- Local directory: --providerType=gdb --path=F:/data/landuse.gdb --table=DLTB
ShapeFile (--providerType=shape)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is shape, indicating the data source is a Shapefile. |
| --path, -P, --url (Required) |
Shapefile directory address, supports local directories and HDFS directories. Note: This address is the folder path containing the Shapefile (.shp, .shx, .dbf, etc.) files. |
Examples:
- Local directory: --providerType=shape --path=E:\data\shp
Elasticsearch (--providerType=elasticsearch)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is elasticsearch, indicating the data source is Elasticsearch. |
| --index (Required) |
Index name. |
| --es.nodes, --nodes (Required) |
ES server address(es). |
| --es.port, --port (Required) |
ES service port. |
| --es.batch.size.bytes, --batch-size-bytes, --BSB | Batch write size, defaults to 1 MB. |
| --es.batch.size.entries, --batch-size-entries, --BSE | Matches es.batch.size.bytes, executes batch updates, defaults to 1000. |
| --es.internal.es.cluster.name, --cluster-name, CN | Cluster name. Required for read/write with cluster versions 5.x and 6.x. Can be set or not for version 7.x. |
| --es.mapping.id | The field/attribute to be written whose name contains es.mapping.id. (Note: Description seems incomplete, typically specifies the field to use as document ID) |
| --number_of_shards | Number of shards, defaults to 5. |
| --number_of_replicas | Number of replicas, defaults to 1. |
Examples:
- --providerType=elasticsearch --index=test --table=test --nodes=localhost --port=9200
SimpleJson (--providerType=simplejson)
| Parameter Name | Parameter Description |
|---|---|
| --providerType (Required) |
The type of data provider. Here it is simplejson, indicating the data source is a SimpleJson file. |
| --path, -P, --url (Required) |
SimpleJson file path or directory address, supports local paths and HDFS paths. |
| --minPartitions, --min-part, -M | Minimum number of Spark RDD partitions, defaults to 0. |
| --metaInfo, --meta-info, --meta | SimpleJson file metadata or metadata address. Users can specify independent SimpleJson file metadata. |
| --idField, --id-field, -IF | Specifies the field for the feature ID. If the SimpleJson object does not have a featureID, you can use --id-fieldto specify a field to construct the featureID, but the user must ensure the field values are unique. |
Examples:
- --providerType=simplejson --path=E:\data\Region.json