Inline Scripts

[thejmazz]

Due to the way we parse a string into a child process it may be the case that certain operations (e.g., a bash for/while read loop) may not run.

It would be nice to use ES6 tagged template literals to define scripts, this could look like:

const myTask = task({
  name: 'my task',
  input: '*.lowercase',
  output: '*.uppercase'
}, ({ input }) => bash`
  input="${input}"
  output="${input.replace(/\.lowercase$/, '.uppercase')}"

  cat $input | awk '{print toupper($0)}' > $output
`)

These could be written to the folder the task is being ran, which has the added benefit of being able to simply cd data/<id> to run the script.

This will involve

• having the tag function return something we can pick up on, so that during the lifecycle hook which creates the operation, it can inject the dir of the task (similar to how if the operationCreator returns a string, it considers it a { shell: true } child process)
• nice way to remove spaces from beginning of lines (regex)
• probably return the spawned child process (as opposed to promise)
• a "base" to add support for other languages (assuming they have unique quirks, file extension, etc)

[thejmazz]

Userland implementation from 9a2e750 for quick reference:

const script = ({ dir, program }) => (strings, ...values) => {
  let content = ''
  strings.forEach((str, i) => {
    content += str + (values[i] || '')
  })

  const scriptLocation = path.resolve(dir, 'script.' + (() => {
    switch (program) {
      case 'bash': return 'sh'
      case 'python': return 'py'
    }
  })())

  fs.writeFileSync(scriptLocation, `
#!/usr/bin/env ${program}
${content.replace(/\n\ {2}/g, '\n')}
  `.trim())

  fs.chmodSync(scriptLocation, '755')

  const cp = spawn(scriptLocation)

  cp.on('error', (err) => console.log(`${scriptLocation} error:`, err))
  cp.on('close', () => console.log(`${scriptLocation} closed`))
  cp.on('exit', () => console.log(`${scriptLocation} exited`))
  cp.on('disconnect', () => console.log(`${scriptLocation} disconnected`))

  cp.stdout.on('data', (chunk) => console.log(`${scriptLocation} stdout: ${chunk}`))
  cp.stderr.on('data', (chunk) => console.log(`${scriptLocation} stderr: ${chunk}`))

  return cp
}

const myTask = task({
  name: 'my task',
  input: '*.lowercase',
  output: '*.uppercase'
}, ({ input, dir }) => script({ dir, program: 'bash' })`
  input="${input}"
  output="${input.replace(/\.lowercase$/, '.uppercase')}"

  cat $input | awk '{print toupper($0)}' > $output
`)