HTCondor

Funnel can be configured to submit workers to HTCondor by making calls to condor_submit.

The Funnel server needs to run on a submission node. Configure Funnel to use HTCondor by including the following config:

It is recommended to update the submit file template so that the funnel worker run command takes a config file as an argument (e.g. funnel worker run --config /opt/funnel_config.yml --taskID {{.TaskId}})

Compute: htcondor

HTCondor:
    Template: |
    universe = vanilla
    getenv = True
    executable = funnel
    arguments = worker run --taskID {{.TaskId}}
    log = {{.WorkDir}}/condor-event-log
    error = {{.WorkDir}}/funnel-stderr
    output = {{.WorkDir}}/funnel-stdout
    should_transfer_files = YES
    when_to_transfer_output = ON_EXIT_OR_EVICT
    {{if ne .Cpus 0 -}}
    {{printf "request_cpus = %d" .Cpus}}
    {{- end}}
    {{if ne .RamGb 0.0 -}}
    {{printf "request_memory = %.0f GB" .RamGb}}
    {{- end}}
    {{if ne .DiskGb 0.0 -}}
    {{printf "request_disk = %.0f GB" .DiskGb}}
    {{- end}}

    queue

The following variables are available for use in the template:

Variable	Description
TaskId	funnel task id
WorkDir	funnel working directory
Cpus	requested cpu cores
RamGb	requested ram
DiskGb	requested free disk space
Zone	requested zone (could be used for queue name)

See https://golang.org/pkg/text/template for information on creating templates.