{"componentChunkName":"component---src-templates-microservices-js","path":"/en/microservices/HDFSDirectoryRegister/","result":{"data":{"markdownRemark":{"html":"<h2 id=\"register-hdfs-directory\"><a href=\"#register-hdfs-directory\" aria-hidden class=\"anchor\"><svg aria-hidden=\"true\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Register HDFS Directory</h2>\n<p>GIS Cloud Suite supports to register internal and external HDFS Directory. The internal one is the built-in HDFS Directory in GIS Cloud Suite. The external one is the HDFS Directory build by users, it is required to configure and register in GIS Cloud Suite. Please follow the steps below to register HDFS Directory.</p>\n<h3 id=\"register-built-in-hdfs-directory\"><a href=\"#register-built-in-hdfs-directory\" aria-hidden class=\"anchor\"><svg aria-hidden=\"true\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Register Built-in HDFS Directory</h3>\n<ol>\n<li>Clicks on <strong>Storage Resources > HDFS Directory</strong> on the left navigation bar.</li>\n<li>\n<p>Clicks <strong>Enable</strong> on the HDFS Directory page.</p>\n<p>\n  <a\n    class=\"gatsby-resp-image-link\"\n    href=\"/iManager_K8S/1014/static/bbfe1f242a490ae85dd8356f9a12b5ef/a13c9/openHDFSen.png\"\n    style=\"display: block\"\n    target=\"_blank\"\n    rel=\"noopener\"\n  >\n  \n  <span\n    class=\"gatsby-resp-image-wrapper\"\n    style=\"position: relative; display: block;  max-width: 840px; margin-left: auto; margin-right: auto;\"\n  >\n    <span\n      class=\"gatsby-resp-image-background-image\"\n      style=\"padding-bottom: 43.333333333333336%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAA7EAAAOxAGVKw4bAAABCElEQVQoz42QzVLDIBSFeRXbUP4CAQIkhcR2HLvIygdw47jRB+nSh/YkmTiOLtpv+LvAOZcLmaZpGIpzru977z0WfqH9R0oJd47Ho28DppIzQRBCWDU4TjFa2yA0xuhbkAyHnKGMMZZSuq6XUt1ELpABjGPrLbfRvl3j+7X7+GpePpXgnAu6cPhFte0wxsjzeXx6LDEEqWplQyon3w1CWylmtVuKFxvYCfO3OPhyiM+lH5K31uKM7h+0koozut/BGxlQmNFmTYXOKfVCGyZotZszO9tAVtc1XH9eSDf+hpRKWnG4tJkdKHEuKAVtjQ/gd8DmJoRxqIFcLq/jOJWhaBfq1eYelMTwDV5MYduuvN99AAAAAElFTkSuQmCC'); background-size: cover; display: block;\"\n    >\n      <img\n        class=\"gatsby-resp-image-image\"\n        style=\"width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;\"\n        alt=\"openHDFS\"\n        title=\"\"\n        src=\"/iManager_K8S/1014/static/bbfe1f242a490ae85dd8356f9a12b5ef/1e088/openHDFSen.png\"\n        srcset=\"/iManager_K8S/1014/static/bbfe1f242a490ae85dd8356f9a12b5ef/65ed1/openHDFSen.png 210w,\n/iManager_K8S/1014/static/bbfe1f242a490ae85dd8356f9a12b5ef/d10fb/openHDFSen.png 420w,\n/iManager_K8S/1014/static/bbfe1f242a490ae85dd8356f9a12b5ef/1e088/openHDFSen.png 840w,\n/iManager_K8S/1014/static/bbfe1f242a490ae85dd8356f9a12b5ef/a13c9/openHDFSen.png 1178w\"\n        sizes=\"(max-width: 840px) 100vw, 840px\"\n      />\n    </span>\n  </span>\n  \n  </a>\n    </p>\n</li>\n<li>After the HDFS Directory environment get started up, clicks on <strong>Console</strong> to enter the HDFS Directory Console.</li>\n<li>\n<p>Clicks the address of hdfs-hue.</p>\n<p>\n  <a\n    class=\"gatsby-resp-image-link\"\n    href=\"/iManager_K8S/1014/static/8a3954c0780a294b290090bcef475bca/705cc/hdfshueen.png\"\n    style=\"display: block\"\n    target=\"_blank\"\n    rel=\"noopener\"\n  >\n  \n  <span\n    class=\"gatsby-resp-image-wrapper\"\n    style=\"position: relative; display: block;  max-width: 840px; margin-left: auto; margin-right: auto;\"\n  >\n    <span\n      class=\"gatsby-resp-image-background-image\"\n      style=\"padding-bottom: 43.80952380952381%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVQoz5VQy46EMAzrt0ybBw2QUhASF077/9+0bitmpNnTmoJMcGKHcN/3dV1mNv8fYa+49v04SinuXmsdH5aOt27wr2LI3RPOOYPaIMDUkR8M/lUMsBKRZXX4M5hqO6IHwhy7qkKN58jlvp7niXYUoQ1iLkzFdCteFiu+1rKVWZEGiegB+jFSJlNzZuKOkCQnVozy7VhLRQJTmTj2EE0PQJeIUopQJs4pvlJCgQPljbITi+ZZs8mUUX/FiNXH38GNKeiHP5wgbiMgYgqkRjAHYhznnXN6gFce3awk1lq4jQt9gbGD9PNBExP/QcI++foB+wX7blg2Pm3ghgAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n    >\n      <img\n        class=\"gatsby-resp-image-image\"\n        style=\"width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;\"\n        alt=\"hdfshueen\"\n        title=\"\"\n        src=\"/iManager_K8S/1014/static/8a3954c0780a294b290090bcef475bca/1e088/hdfshueen.png\"\n        srcset=\"/iManager_K8S/1014/static/8a3954c0780a294b290090bcef475bca/65ed1/hdfshueen.png 210w,\n/iManager_K8S/1014/static/8a3954c0780a294b290090bcef475bca/d10fb/hdfshueen.png 420w,\n/iManager_K8S/1014/static/8a3954c0780a294b290090bcef475bca/1e088/hdfshueen.png 840w,\n/iManager_K8S/1014/static/8a3954c0780a294b290090bcef475bca/705cc/hdfshueen.png 1071w\"\n        sizes=\"(max-width: 840px) 100vw, 840px\"\n      />\n    </span>\n  </span>\n  \n  </a>\n    </p>\n</li>\n<li>Fills in the username and password to create an account.</li>\n<li>Clicks on <strong>File Browser</strong> to enter your folder.</li>\n<li>\n<p>Use <strong>Upload</strong> button to upload data.</p>\n<blockquote>\n<p>Notes:  </p>\n<p>HDFS Directory is different to others storage resource, it does not support to register first, then upload data through Data Catalog Service. The HDFS Directory needs to upload data through the Hue, then register to Data Catalog Service.</p>\n</blockquote>\n</li>\n<li>\n<p>Back to the HDFS Directory register page, click on <strong>Register HDFS Directory</strong>. Refer to the introductions in <strong>Register External HDFS Directory</strong> to fill out the information except HDFS directory option.</p>\n<ul>\n<li>HDFS directory: Fill in the path of hdfs-namenode and the path of account. For example, the path of hdfs-namenode is ‘hdfs://39.106.103.12:30691’ in the screenshot above. The path of the account is <code class=\"gatsby-code-text\">/user/username</code>, <code class=\"gatsby-code-text\">/user/</code> is the fixed default path, <code class=\"gatsby-code-text\">username</code> is the username you created in step 5. The HDFS directory should be ‘hdfs://39.106.103.12:30691/user/username’.</li>\n</ul>\n</li>\n</ol>\n<h3 id=\"register-external-hdfs-directory\"><a href=\"#register-external-hdfs-directory\" aria-hidden class=\"anchor\"><svg aria-hidden=\"true\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Register External HDFS Directory</h3>\n<ol>\n<li>Clicks on <strong>Storage Resources > HDFS Directory</strong> on the left navigation bar.</li>\n<li>Clicks on <strong>Register HDFS Directory</strong>.</li>\n<li>\n<p>Fills out the following information to register the HDFS Directory.</p>\n<p>\n  <a\n    class=\"gatsby-resp-image-link\"\n    href=\"/iManager_K8S/1014/static/4f5665a7bacddecf1b55166dcb8a6a3e/302a4/addHDFStocomputelisten.png\"\n    style=\"display: block\"\n    target=\"_blank\"\n    rel=\"noopener\"\n  >\n  \n  <span\n    class=\"gatsby-resp-image-wrapper\"\n    style=\"position: relative; display: block;  max-width: 840px; margin-left: auto; margin-right: auto;\"\n  >\n    <span\n      class=\"gatsby-resp-image-background-image\"\n      style=\"padding-bottom: 74.28571428571428%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAIAAABr+ngCAAAACXBIWXMAAA7EAAAOxAGVKw4bAAABPElEQVQoz5WT3VbDIAzH+/4v4It45a1v4PHGC89x0ws7Rr9DIJBgWLdanU73PylQ2h+EJFRN0wxDP+rT9wAT6DNNIYT4u4hIRLquq+p6Z8x+v7faT+ASC8UUI8d0ZvFoREkkd11fgcOcM0uWnFNK6D1pdwTS2nRythiTIu0CFwlTCIh4+BxFuMytlE+ax+M4fsLC7GBCpzAhOj0YM1+AhwXW/3Q3B04FgZO+c3F0zVyCvfeHIJOBTJwv6Ds8r08UNGDy1du/3T7C3qvnfAWMnlOE+bQAGrCrdy7bRg0yarbeR6F0zZkpaGoULB2SlBL6z85akjrSgtakT0WgSVqqg1daVtGxIv0wVOjDasnSPuzy7XNuUE4TP7tdKmz7+matLTejtHZo7d2jubk3T9vamrpcmTMZY9q2fdlsPgDLwG7UeRk1UAAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n    >\n      <img\n        class=\"gatsby-resp-image-image\"\n        style=\"width: 100%; height: 100%; margin: 0; vertical-align: middle; position: absolute; top: 0; left: 0; box-shadow: inset 0px 0px 0px 400px white;\"\n        alt=\"addHDFStocomputelist\"\n        title=\"\"\n        src=\"/iManager_K8S/1014/static/4f5665a7bacddecf1b55166dcb8a6a3e/1e088/addHDFStocomputelisten.png\"\n        srcset=\"/iManager_K8S/1014/static/4f5665a7bacddecf1b55166dcb8a6a3e/65ed1/addHDFStocomputelisten.png 210w,\n/iManager_K8S/1014/static/4f5665a7bacddecf1b55166dcb8a6a3e/d10fb/addHDFStocomputelisten.png 420w,\n/iManager_K8S/1014/static/4f5665a7bacddecf1b55166dcb8a6a3e/1e088/addHDFStocomputelisten.png 840w,\n/iManager_K8S/1014/static/4f5665a7bacddecf1b55166dcb8a6a3e/302a4/addHDFStocomputelisten.png 1080w\"\n        sizes=\"(max-width: 840px) 100vw, 840px\"\n      />\n    </span>\n  </span>\n  \n  </a>\n    </p>\n<ul>\n<li>Storage ID(Required): The name of the Storage resource, the name is customizable.</li>\n<li>Data storage type(Required): HDFS Directory.</li>\n<li>\n<p>HDFS directory:</p>\n<ul>\n<li>If the object to register is a csv file, you could fill in the path of HDFS plus csv name, such as <code class=\"gatsby-code-text\">hdfs://{ip}:9000/data/newyork_taxi_2013-01_14k.csv</code>.</li>\n<li>\n<p>If the object to register is the directory with csv files, the csv folder has many CSV files:</p>\n<ul>\n<li>If all the csv files have a same format, such as fields and attributes. Fill in the path of the parent directory of csvfolder. For example, <code class=\"gatsby-code-text\">hdfs://{ip}:9000/data</code>, <code class=\"gatsby-code-text\">data</code> is the parent directory of csvfolder.</li>\n<li>If the format of csv files are different. Fill in the directory of csv files, such as <code class=\"gatsby-code-text\">hdfs://{ip}:9000/data/csvfolder</code>.</li>\n</ul>\n</li>\n<li>If the object to register is the directory with index files. Fill in the directory of index files, such as <code class=\"gatsby-code-text\">hdfs://{ip}:9000/data/indexfolder</code>, <code class=\"gatsby-code-text\">indexfolder</code> has index files inside.</li>\n<li>If the registered HDFS cluster is high availability mode. Fill in the address of high availability HDFS, such as <code class=\"gatsby-code-text\">hdfs://{HA address}/data</code>.</li>\n</ul>\n</li>\n<li>\n<p>Open Kerberos certification: (If you open Kerberos certification, please fiil out the following information)</p>\n<ul>\n<li>Subject name: The format is username@domain, such as iserver@SUPERMAP.COM. The username must be a system user in HDFS master node. The domain is set by Kerberos service, and must be the same as domain of HDFS cluster.</li>\n<li>Key file path: The path of the key files in GIS Cloud Suite host machine. The key files were generated when building the cluster. You need to copy the key files to GIS Cloud Suite host machine.</li>\n<li>Kerberos client configuration file path: The path of the Kerberos client configuration files in GIS Cloud Suite host machine.</li>\n</ul>\n</li>\n<li>\n<p>High availability mode(If you open HA mode, please fill out the following information):</p>\n<ul>\n<li>Hadoop cluster cor-site.xml path: The path of the Hadoop cluster core-site.xml file in GIS Cloud Suite host machine.</li>\n<li>HDFS cluster hdfs-site.xml path: The path of the HDFS cluster hdfs-site.xml file in GIS Cloud Suite host machine.</li>\n</ul>\n</li>\n</ul>\n</li>\n<li>\n<p>Clicks on <strong>Register</strong>, the new added HDFS would be displayed in the list.</p>\n<blockquote>\n<p>Notes:  </p>\n<p>More details of HDFS Directory please see SuperMap iServer Documentation.</p>\n</blockquote>\n</li>\n</ol>","frontmatter":{"title":"GIS Cloud Suite","next":null,"prev":null},"fields":{"path":"content/microservices/HDFSDirectoryRegister.en.md","slug":"/en/microservices/HDFSDirectoryRegister/","langKey":"en"}}},"pageContext":{"slug":"/en/microservices/HDFSDirectoryRegister/"}}}