|
1 | | -# cloudwatcher |
2 | | -File system notification for Cloud platforms in Golang. |
| 1 | +# Cloudwatcher |
| 2 | +    |
| 3 | + |
| 4 | +File system notification for Cloud platforms (and not) in Golang. |
| 5 | + |
| 6 | +cloudwatcher is a file system notification library for cloud platforms (and not) in Go. |
| 7 | +Currently it implements the watchers for the following services: |
| 8 | +- Amazon S3 |
| 9 | +- Google Drive |
| 10 | +- Dropbox |
| 11 | +- local filesystem (fsnotify/polling) |
| 12 | + |
| 13 | +It is possible specify the directory to monitor and the polling time (how much often the watcher should check that directory), |
| 14 | +and every time a new file is added, removed or changed, an event is generated and sent to the `Events` channel. |
| 15 | + |
| 16 | +## Usage |
| 17 | + |
| 18 | +```go |
| 19 | +package main |
| 20 | + |
| 21 | +import ( |
| 22 | + "fmt" |
| 23 | + "time" |
| 24 | + |
| 25 | + "github.com/Matrix86/cloudwatcher" |
| 26 | +) |
| 27 | + |
| 28 | +func main() { |
| 29 | + s, err := cloudwatcher.New("local", "/home/user/tests", time.Second) |
| 30 | + if err != nil { |
| 31 | + fmt.Printf("ERROR: %s", err) |
| 32 | + return |
| 33 | + } |
| 34 | + |
| 35 | + config := map[string]string{ |
| 36 | + "disable_fsnotify": "false", |
| 37 | + } |
| 38 | + |
| 39 | + err = s.SetConfig(config) |
| 40 | + if err != nil { |
| 41 | + fmt.Printf("ERROR: %s", err) |
| 42 | + return |
| 43 | + } |
| 44 | + |
| 45 | + err = s.Start() |
| 46 | + defer s.Close() |
| 47 | + for { |
| 48 | + select { |
| 49 | + case v := <-s.GetEvents(): |
| 50 | + fmt.Printf("EVENT: %s %s\n", v.Key, v.TypeString()) |
| 51 | + |
| 52 | + case e := <-s.GetErrors(): |
| 53 | + fmt.Printf("ERROR: %s\n", e) |
| 54 | + } |
| 55 | + } |
| 56 | +} |
| 57 | +``` |
| 58 | + |
| 59 | +The channel returned by `GetEvents()` function, will return an [Event](event.go) struct that contains the event type, the Key |
| 60 | +with the name of the file that generates the event and the object itself. |
| 61 | + |
| 62 | +> :warning: check the Event.Object field before use it...in some cases it could be nil (FileDelete event with fsnotify) |
| 63 | +
|
| 64 | +## Amazon S3 |
| 65 | + |
| 66 | +The config of the S3 watcher is the following: |
| 67 | + |
| 68 | +```go |
| 69 | +config := map[string]string{ |
| 70 | + "bucket_name": "storage", |
| 71 | + "endpoint": "s3-us-west-2.amazonaws.com", |
| 72 | + "access_key": "user", |
| 73 | + "secret_key": "secret", |
| 74 | + "token": "", |
| 75 | + "region": "us-west-2", |
| 76 | + "ssl_enabled": "true", |
| 77 | +} |
| 78 | +``` |
| 79 | + |
| 80 | +An example can be found [here](examples/s3/s3.go). |
| 81 | + |
| 82 | +> :gem: [minio](https://docs.min.io/docs/minio-quickstart-guide.html) can be used for testing purposes |
| 83 | +
|
| 84 | +## Google Drive |
| 85 | + |
| 86 | +In order to use it you need to enable the drive API from [here](https://developers.google.com/drive/api/v3/enable-drive-api) . |
| 87 | +After that you can specify the `client-id` and the `client-secret` on the config file. |
| 88 | +The logic to retrieve the token has to be handled outside the library and the `token` field should contain the json. |
| 89 | + |
| 90 | +You can find an example in the [examples directory](examples/gdrive/gdrive.go). |
| 91 | + |
| 92 | +```go |
| 93 | +config := map[string]string{ |
| 94 | + "debug": "true", |
| 95 | + "token": Token, |
| 96 | + "client_id": ClientId, |
| 97 | + "client_secret": ClientSecret, |
| 98 | +} |
| 99 | +``` |
| 100 | + |
| 101 | +## Dropbox |
| 102 | + |
| 103 | +First, you need to register a new app from the [developer console](https://www.dropbox.com/developers/). |
| 104 | +Use the `client-id` and the `client-secret` to retrieve the user token and use it on the `token` field of the config. |
| 105 | + |
| 106 | +You can find an example in the [examples directory](examples/dropbox/dropbox.go). |
| 107 | + |
| 108 | +```go |
| 109 | +config := map[string]string{ |
| 110 | + "debug": "true", |
| 111 | + "token": Token, |
| 112 | +} |
| 113 | +``` |
| 114 | + |
| 115 | +## Local filesystem |
| 116 | + |
| 117 | +It is based on [fsnotify](https://github.com/fsnotify/fsnotify) library, so it is cross platform: Windows, Linux, BSD and macOS. |
| 118 | +It is not mandatory to call the `SetConfig()` function, and the polling time argument of `cloudwatcher.New` is not used. |
| 119 | + |
| 120 | +Setting `disable_fsnotify` parameter on config to "true" the watcher doesn't use fsnotify and use the listing approach instead. |
| 121 | + |
| 122 | +> :warning: not set `disable_fsnotify` to "true" if you plan to use it on a big directory!!! It could increase the I/O on disk |
0 commit comments