Skip to content

Commit a8f53a7

Browse files
author
R. S. Doiel
committed
fixing bad merge
2 parents b8aed98 + 6947bd1 commit a8f53a7

8 files changed

Lines changed: 308 additions & 105 deletions

File tree

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: CodeMeta2CFF
2+
run-name: Run CodeMeta2CFF after ${{github.event_name}} by ${{github.actor}}
3+
4+
on:
5+
push:
6+
paths: ['codemeta.json']
7+
workflow_dispatch:
8+
inputs:
9+
reason:
10+
description: 'Reason'
11+
required: false
12+
default: 'Manual trigger'
13+
14+
jobs:
15+
CodeMeta2CFF:
16+
runs-on: ubuntu-latest
17+
permissions:
18+
contents: write
19+
steps:
20+
- name: Checkout
21+
uses: actions/checkout@v4
22+
- name: Convert CFF
23+
uses: caltechlibrary/codemeta2cff@main
24+
- name: Commit CFF
25+
uses: EndBug/add-and-commit@v9
26+
with:
27+
message: 'Add updated CITATION.cff from codemeta.json file'
28+
add: 'CITATION.cff'

CITATION.cff

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +0,0 @@
1-
cff-version: 1.2.0
2-
message: "If you use this software, please cite it as below."
3-
type: software
4-
title: "datatools"
5-
abstract: "A set of command line tools for working with CSV, Excel
6-
Workbooks, JSON and structured text documents."
7-
authors:
8-
- family-names: Doiel
9-
given-names: R. S.
10-
orcid: ""
11-
12-
maintainers:
13-
- family-names: Doiel
14-
given-names: R. S.
15-
orcid: ""
16-
17-
repository-code: "https://github.com/caltechlibrary/datatools"
18-
version: 1.3.5
19-
license-url: "https://data.caltech.edu/license"
20-
keywords: [ "csv", "excel", "sql", "json", "yaml", "xlsx", "golang", "bash" ]
21-
date-released: 2026-02-12

README.md

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,13 @@ datatools
44

55
_datatools_ is a rich collection of command line programs targetting
66
data conversion, cleanup and analysis directly from your favorite
7-
POSIX shell. It has proven useful for data collaberations where
7+
POSIX shell. It has proven useful for data collaborations where
88
individual members of a project may prefer different toolsets in their
99
analysis (e.g. Julia, R, Python) but want to work from a common baseline.
1010
It also has been used intensively for internal reporting from various
1111
Caltech Library metadata sources.
1212

13-
The tools fall into three broad categories
13+
The tools fall into three broad categories
1414

1515
- data transformation and conversion
1616
- shell scripting helpers
@@ -39,7 +39,7 @@ intermediate data format.
3939
For the shell
4040
-------------
4141

42-
Various utilities for simplifying work on the command line.
42+
Various utilities for simplifying work on the command line.
4343

4444
+ [findfile](docs/findfile/) - find files based on prefix, suffix or contained string
4545
+ [finddir](docs/finddir/) - find directories based on prefix, suffix or contained string
@@ -53,15 +53,15 @@ Various utilities for simplifying work on the command line.
5353
For strings
5454
-----------
5555

56-
_datatools_ provides the [string](docs/string/) command for working with
57-
text strings (limited to memory available). This is commonly needed when
58-
cleanup data for analysis. The _string_ command was created for when the
59-
old Unix standbys- grep, awk, sed, tr are unwieldly or inconvient.
60-
_string_ provides operations are common in most language like, trimming,
61-
spliting, and transforming letter case. The _string_ command also makes
62-
it easy to join JSON string arrays into single a string using a delimiter
63-
or split a string into a JSON array based on a delimiter. The form of the
64-
command is `string [OPTIONS] [ACTION] [ARCTION_PARAMETERS...]`
56+
_datatools_ provides the [string](docs/string/) command for working with
57+
text strings (limited to memory available). This is commonly needed when
58+
cleanup data for analysis. The _string_ command was created for when the
59+
old Unix standbys- grep, awk, sed, tr are unwieldly or inconvenient.
60+
_string_ provides operations are common in most language like, trimming,
61+
splitting, and transforming letter case. The _string_ command also makes
62+
it easy to join JSON string arrays into single a string using a delimiter
63+
or split a string into a JSON array based on a delimiter. The form of the
64+
command is `string [OPTIONS] [ACTION] [ACTION_PARAMETERS...]`
6565

6666
```shell
6767
string toupper "one two three"
@@ -82,6 +82,5 @@ See [string](docs/string/) for full details
8282
Installation
8383
------------
8484

85-
See [INSTALL.md](install.html) for details for installing pre-compiled
85+
See [INSTALL.md](https://caltechlibrary.github.io/datatools/INSTALL.html) for details for installing pre-compiled
8686
versions of the programs.
87-

cmd/reldocpath/reldocpath.go

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
//
2+
// reldocpath.go takes a source document path and a target document path with same base path
3+
// returning a relative path to the target file.
4+
//
5+
// @author R. S. Doiel, <rsdoiel@caltech.edu>
6+
//
7+
// Copyright (c) 2021, Caltech
8+
// All rights not granted herein are expressly reserved by Caltech.
9+
//
10+
// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
11+
//
12+
// 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
13+
//
14+
// 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
15+
//
16+
// 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
17+
//
18+
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
19+
//
20+
package main
21+
22+
import (
23+
"flag"
24+
"fmt"
25+
"os"
26+
"path"
27+
28+
// My packages
29+
"github.com/caltechlibrary/datatools"
30+
)
31+
32+
var (
33+
helpText = `%{app_name}(1) user manual | version {version} {release_hash}
34+
% R. S. Doiel
35+
% {release_date}
36+
37+
# NAME
38+
39+
{app_name}
40+
41+
# SYNOPSIS
42+
43+
{app_name} [OPTIONS] [STRING]
44+
45+
# DESCRIPTION
46+
47+
Given a source document path, a target document path calculate and
48+
the implied common base path calculate the relative path for target.
49+
50+
# EXAMPLE
51+
52+
Given
53+
54+
reldocpath chapter-01/lesson-03.html css/site.css
55+
56+
would output
57+
58+
.../css/site.css
59+
60+
`
61+
62+
// Standard options
63+
showHelp bool
64+
showVersion bool
65+
showLicense bool
66+
showExamples bool
67+
)
68+
69+
func main() {
70+
appName := path.Base(os.Args[0])
71+
version := datatools.Version
72+
license := datatools.LicenseText
73+
releaseDate := datatools.ReleaseDate
74+
releaseHash := datatools.ReleaseHash
75+
76+
// Standard Options
77+
flag.BoolVar(&showHelp, "help", showHelp, "display help")
78+
flag.BoolVar(&showLicense, "license", showLicense, "display license")
79+
flag.BoolVar(&showVersion, "version", showVersion, "display version")
80+
81+
// Parse environment and options
82+
flag.Parse()
83+
args := flag.Args()
84+
85+
//in := os.Stdin
86+
out := os.Stdout
87+
eout := os.Stderr
88+
89+
// Handle the default options
90+
if showHelp {
91+
fmt.Fprintf(out, "%s\n", datatools.FmtHelp(helpText, appName, version, releaseDate, releaseHash))
92+
os.Exit(0)
93+
}
94+
if showLicense == true {
95+
fmt.Fprintf(out, "%s\n", license)
96+
os.Exit(0)
97+
}
98+
if showVersion == true {
99+
fmt.Fprintf(out, "datatools, %s %s %s\n", appName, version, releaseHash)
100+
os.Exit(0)
101+
}
102+
103+
if len(args) != 2 {
104+
fmt.Fprintf(eout, "Expected a source and target file path\n For help try: %s -help", appName)
105+
os.Exit(1)
106+
}
107+
source, target := args[0], args[1]
108+
fmt.Fprintf(out, `%s`, datatools.RelativeDocPath(source, target))
109+
}

cmd/urlencode/urlencode.go

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
//
2+
// urlencode.go is a simple command line utility to encode a string in a URL friendly way.
3+
//
4+
// @author R. S. Doiel, <rsdoiel@caltech.edu>
5+
//
6+
// Copyright (c) 2021, Caltech
7+
// All rights not granted herein are expressly reserved by Caltech.
8+
//
9+
// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
10+
//
11+
// 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
12+
//
13+
// 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
14+
//
15+
// 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
16+
//
17+
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
18+
//
19+
package main
20+
21+
import (
22+
"flag"
23+
"fmt"
24+
"io/ioutil"
25+
"net/url"
26+
"path"
27+
"os"
28+
"strings"
29+
30+
// CaltechLibrary Packages
31+
"github.com/caltechlibrary/datatools"
32+
)
33+
34+
var (
35+
helpText = `%{app_name}(1) user manual | version {version} {release_hash}
36+
% R. S. Doiel
37+
% {release_date}
38+
39+
# NAME
40+
41+
{app_name}
42+
43+
# SYNOPSIS
44+
45+
{app_name} [OPTIONS] [STRING]
46+
47+
# DESCRIPTION
48+
49+
{app_name} is a simple command line utility to URL encode content. By default
50+
it reads from standard input and writes to standard out. You can
51+
also specifty the string to encode as a command line parameter.
52+
53+
You can provide the string to encode as a command line parameter otherwise it
54+
will be read from standard input.
55+
56+
# OPTIONS
57+
58+
-help
59+
: display help
60+
61+
-license
62+
: display license
63+
64+
-version
65+
: display version
66+
67+
-query
68+
: Use URL query style encoding (use plus for spaces)
69+
70+
-newline
71+
: Append a trailing newline
72+
73+
# EXAMPLE
74+
75+
echo "This is the string to encode & nothing else!" | urlencode
76+
77+
would yield
78+
79+
This%20is%20the%20string%20to%20encode%20&%20nothing%20else%0A
80+
81+
`
82+
// Standard Options
83+
showHelp bool
84+
showLicense bool
85+
showVersion bool
86+
87+
// App Options
88+
useQueryEscape bool
89+
newLine bool
90+
)
91+
92+
func main() {
93+
appName := path.Base(os.Args[0])
94+
version := datatools.Version
95+
license := datatools.LicenseText
96+
releaseDate := datatools.ReleaseDate
97+
releaseHash := datatools.ReleaseHash
98+
99+
// Standard Options
100+
flag.BoolVar(&showHelp, "help", showHelp, "display help")
101+
flag.BoolVar(&showLicense, "license", showLicense, "display license")
102+
flag.BoolVar(&showVersion, "version", showVersion, "display version")
103+
flag.BoolVar(&newLine, "newline", false, "append a newline character")
104+
flag.BoolVar(&useQueryEscape, "query", false, "use query escape (pluses for spaces)")
105+
106+
// Parse environment and options
107+
flag.Parse()
108+
args := flag.Args()
109+
110+
in := os.Stdin
111+
out := os.Stdout
112+
eout := os.Stderr
113+
114+
// Handle the default options
115+
if showHelp {
116+
fmt.Fprintf(out, "%s\n", datatools.FmtHelp(helpText, appName, version, releaseDate, releaseHash))
117+
os.Exit(0)
118+
}
119+
if showLicense == true {
120+
fmt.Fprintf(out, "%s\n", license)
121+
os.Exit(0)
122+
}
123+
if showVersion == true {
124+
fmt.Fprintf(out, "datatools, %s %s %s\n", appName, version, releaseHash)
125+
os.Exit(0)
126+
}
127+
128+
nl := "\n"
129+
if newLine == false {
130+
nl = ""
131+
}
132+
133+
var src string
134+
135+
if len(args) > 0 {
136+
src = strings.Join(args, " ")
137+
} else {
138+
buf, err := ioutil.ReadAll(in)
139+
if err != nil {
140+
fmt.Fprintf(eout, "%s\n", err)
141+
os.Exit(1)
142+
}
143+
src = fmt.Sprintf("%s", buf)
144+
}
145+
if useQueryEscape == true {
146+
fmt.Fprintf(out, "%s%s", url.QueryEscape(src), nl)
147+
} else {
148+
fmt.Fprintf(out, "%s%s", url.PathEscape(src), nl)
149+
}
150+
}

codemeta.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44
"codeRepository": "https://github.com/caltechlibrary/datatools",
55
"author": [
66
{
7-
"id": "https://orcid.org/0000-0003-0900-6903",
8-
"type": "Person",
7+
"@id": "https://orcid.org/0000-0003-0900-6903",
8+
"@type": "Person",
99
"givenName": "R. S.",
1010
"familyName": "Doiel",
1111
"affiliation": {
@@ -17,8 +17,8 @@
1717
],
1818
"maintainer": [
1919
{
20-
"id": "https://orcid.org/0000-0003-0900-6903",
21-
"type": "Person",
20+
"@id": "https://orcid.org/0000-0003-0900-6903",
21+
"@type": "Person",
2222
"givenName": "R. S.",
2323
"familyName": "Doiel",
2424
"affiliation": {
@@ -66,4 +66,4 @@
6666
"issueTracker": "https://github.com/caltechlibrary/datatools/issues",
6767
"downloadUrl": "https://github.com/caltechlibrary/datatools/releases/",
6868
"releaseNotes": "Added urlencode, urldecode and reldocpath cli."
69-
}
69+
}

0 commit comments

Comments
 (0)