<section id="spark-dagster-spark">
<h1>Spark (dagster-spark)<a class="headerlink" href="#spark-dagster-spark" title="Permalink to this heading">¶</a></h1>
<dl class="py class">
<dt class="sig sig-object py" id="dagster_spark.SparkOpError">
class dagster_spark.SparkOpError<a class="reference internal" href="../../../../_modules/dagster_spark/types/#SparkOpError">[source]</a><a class="headerlink" href="#dagster_spark.SparkOpError" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="dagster_spark.define_spark_config">
dagster_spark.define_spark_config()<a class="reference internal" href="../../../../_modules/dagster_spark/configs/#define_spark_config">[source]</a><a class="headerlink" href="#dagster_spark.define_spark_config" title="Permalink to this definition">¶</a></dt>
<dd>Spark configuration.
<dl class="simple">
<dt>See the Spark documentation for reference:</dt><dd><a class="reference external" href="https://spark.apache.org/docs/latest/submitting-applications.html">https://spark.apache.org/docs/latest/submitting-applications.html</a>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="dagster_spark.create_spark_op">
dagster_spark.create_spark_op(name, main_class, description=None, required_resource_keys=frozenset({'spark'}))<a class="reference internal" href="../../../../_modules/dagster_spark/ops/#create_spark_op">[source]</a><a class="headerlink" href="#dagster_spark.create_spark_op" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="dagster_spark.construct_spark_shell_command">
dagster_spark.construct_spark_shell_command(application_jar, main_class, master_url=None, spark_conf=None, deploy_mode=None, application_arguments=None, spark_home=None)<a class="reference internal" href="../../../../_modules/dagster_spark/utils/#construct_spark_shell_command">[source]</a><a class="headerlink" href="#dagster_spark.construct_spark_shell_command" title="Permalink to this definition">¶</a></dt>
<dd>Constructs the spark-submit command for a Spark job.
</dd></dl>

<dl class="py data">
<dt class="sig sig-object py" id="dagster_spark.spark_resource">
dagster_spark.spark_resource ResourceDefinition<a class="reference internal" href="../../../../_modules/dagster_spark/resources/#spark_resource">[source]</a><a class="headerlink" href="#dagster_spark.spark_resource" title="Permalink to this definition">¶</a></dt>
<dd><div class="line-block">
<div class="line"> </div>
</div>
</dd></dl>

</section>

You are viewing an outdated version of the documentation.

Spark (dagster-spark)¶