]> Git — Sourcephile - gargantext.git/blob - README.md
[VERSION] +1 to 0.0.6.9.9.4.3
[gargantext.git] / README.md
1 <div align="center"><img height="180" src="https://gitlab.iscpif.fr/gargantext/main/raw/master/images/logo.png"></div>
2
3 &nbsp;
4 # Gargantext with Haskell (Backend instance)
5
6 ![Haskell](https://img.shields.io/badge/Code-Haskell-informational?style=flat&logo=haskell&color=6144b3)&nbsp;&nbsp;![Stack](https://img.shields.io/badge/Tools-Stack-informational?style=flat&logo=&color=6144b3)&nbsp;&nbsp;![GHC](https://img.shields.io/badge/Tools-GHC-informational?style=flat&logo=&color=2E677B)&nbsp;&nbsp;![Nix](https://img.shields.io/badge/Package%20manager-Nix-informational?style=flat&logo=debian&color=6586c8)&nbsp;&nbsp;![Docker](https://img.shields.io/badge/Tools-Docker-informational?style=flat&logo=docker&color=003f8c)
7
8 #### Table of Contents
9 1. [About the project](#about)
10 2. [Installation](#install)
11 3. [Initialization](#init)
12 4. [Launch & develop GarganText](#launch)
13 5. [Uses cases](#use-cases)
14 6. [GraphQL](#graphql)
15 7. [PostgreSQL](#postgresql)
16
17 ## About the project <a name="about"></a>
18
19 GarganText is a collaborative web-decentralized-based macro-service
20 platform for the exploration of unstructured texts. It combines tools
21 from natural language processing, text-data-mining bricks, complex
22 networks analysis algorithms and interactive data visualization tools
23 to pave the way toward new kinds of interactions with your textual and
24 digital corpora.
25
26 This software is free (as "Libre" in French) software, developed by the
27 CNRS Complex Systems Institute of Paris Île-de-France (ISC-PIF) and its
28 partners.
29
30 GarganText Project: this repo builds the backend for the frontend server built by
31 [backend](https://gitlab.iscpif.fr/gargantext/haskell-gargantext).
32
33
34 ## Installation <a name="install"></a>
35
36 Disclaimer: since this project is still in development, this document
37 remains in progress. Please report and improve this documentation if you
38 encounter any issues.
39
40 #### Prerequisite
41
42 Clone the project.
43 ```shell
44 git clone https://gitlab.iscpif.fr/gargantext/haskell-gargantext.git
45 cd haskell-gargantext
46 ```
47 #### 1. Install Stack
48
49 Install [Stack (or Haskell Tool Stack)](https://docs.haskellstack.org/en/stable/):
50
51 ```shell
52 curl -sSL https://get.haskellstack.org/ | sh
53 ```
54
55 Verify the installation is complete with
56 ```shell
57 stack --version
58 Version 2.9.1
59 ```
60
61 #### 2. Install Nix
62
63 Install [Nix](https://nixos.org/download.html):
64
65 ```shell
66 $ sh <(curl -L https://nixos.org/nix/install) --daemon
67 ```
68
69 Verify the installation is complete with
70 ```shell
71 $ nix-env --version
72 nix-env (Nix) 2.11.0
73 ```
74
75 > **NOTE INFO (upgrade/downgrade if needed)**
76 > Gargantext works with Nix 2.11.0 (older version than current 2.13.2). To downgrade your Nix version:
77 >
78 > `nix-channel --update; nix-env -iA nixpkgs.nixVersions.nix_2_11 nixpkgs.cacert; systemctl daemon-reload; systemctl restart nix-daemon`
79 >
80 > Upgrading Nix: https://nixos.org/manual/nix/unstable/installation/upgrading.html
81 >
82 > **Then, don't forget to exit Terminal and reload to take into account the version change**
83
84
85 #### 3. Build Core Code
86
87 NOTE: Default build (with optimizations) requires large amounts of RAM
88 (16GB at least). To avoid heavy compilation times and swapping out your
89 machine, it is recommended to `stack build` with the `--fast` flag,
90 i.e.:
91
92 ``` sh
93 stack --nix build --fast
94 ```
95
96 If the build is finishing without error, you are ready to launch
97 GarganText! See next step.
98
99 &nbsp;
100
101
102 ## Initialization <a name="init"></a>
103
104 #### 1. Docker-compose will configure your database and some NLP bricks (such as CoreNLP):
105
106 ``` sh
107 # If docker is not installed:
108 # curl -sSL https://gitlab.iscpif.fr/gargantext/haskell-gargantext/raw/dev/devops/docker/docker-install | sh
109 cd devops/docker
110 docker compose up
111 ```
112 Initialization schema should be loaded automatically (from `devops/postgres/schema.sql`).
113
114 #### 2. Then install:
115 ``` sh
116 stack --nix install
117 ```
118
119 #### 3. Copy the configuration file:
120 ``` sh
121 cp gargantext.ini_toModify gargantext.ini
122 ```
123 > Do not worry, `.gitignore` avoids adding this file to the repository by mistake, then you can change the passwords in gargantext.ini safely.
124
125 #### 4. A user have to be created first as instance:
126 ``` sh
127 ~/.local/bin/gargantext-init "gargantext.ini"
128 ```
129 Now, `user1` is created with password `1resu`
130
131 #### 5. Clone FRONTEND repository:
132
133 From the Backend root folder (haskell-gargantext):
134
135 ```shell
136 git clone ssh://git@gitlab.iscpif.fr:20022/gargantext/purescript-gargantext.git
137 ```
138 &nbsp;
139
140 ## Launch & develop GarganText <a name="launch"></a>
141
142 From the Backend root folder (haskell-gargantext):
143
144 ``` shell
145 ./start
146 ```
147
148 > The start script runs following commands:
149 > `docker compose up` to run the Docker for postgresql from devops/docker folder
150 > `stack --nix exec gargantext-server -- --ini gargantext.ini --run Prod` to run other services
151
152 For frontend development and compilation, see the [Frontend Readme.md](https://gitlab.iscpif.fr/gargantext/purescript-gargantext#dev)
153
154
155 ## Use Cases <a name="use-cases"></a>
156
157 ### Multi-User with Graphical User Interface (Server Mode)
158
159 ``` sh
160 ~/.local/bin/stack --docker exec gargantext-server -- --ini "gargantext.ini" --run Prod
161 ```
162
163 Then you can log in with `user1` / `1resu`
164
165
166 ### Command Line Mode tools
167
168 #### Simple cooccurrences computation and indexation from a list of Ngrams
169
170 ``` sh
171 stack --docker exec gargantext-cli -- CorpusFromGarg.csv ListFromGarg.csv Ouput.json
172 ```
173
174 ### Analyzing the ngrams table repo
175
176 We store the repository in directory `repos` in the [CBOR](https://cbor.io/)
177 file format. To decode it to JSON and analyze, say, using
178 [jq](https://shapeshed.com/jq-json/), use the following command:
179
180 ``` sh
181 cat repos/repo.cbor.v5 | stack --nix exec gargantext-cbor2json | jq .
182 ```
183 ### Documentation
184
185 To build documentation, run:
186
187 ```sh
188 stack --nix build --haddock --no-haddock-deps --fast
189 ```
190
191 (in `.stack-work/dist/x86_64-linux-nix/Cabal-3.2.1.0/doc/html/gargantext`).
192
193 ## GraphQL <a name="graphql"></a>
194
195 Some introspection information.
196
197 Playground is located at http://localhost:8008/gql
198
199 ### List all GraphQL types in the Playground
200
201 ```
202 {
203 __schema {
204 types {
205 name
206 }
207 }
208 }
209 ```
210
211 ### List details about a type in GraphQL
212
213 ```
214 {
215 __type(name:"User") {
216 fields {
217 name
218 description
219 type {
220 name
221 }
222 }
223 }
224 }
225 ```
226 ## PostgreSQL <a name="pgsql"></a>
227
228 ### Upgrading using Docker
229
230 https://www.cloudytuts.com/tutorials/docker/how-to-upgrade-postgresql-in-docker-and-kubernetes/
231
232 To upgrade PostgreSQL in Docker containers, for example from 11.x to 14.x, simply run:
233 ```sh
234 docker exec -it <container-id> pg_dumpall -U gargantua > 11-db.dump
235 ```
236
237 Then, shut down the container, replace `image` section in
238 `devops/docker/docker-compose.yaml` with `postgres:14`. Also, it is a good practice to create a new volume, say `garg-pgdata14` and bind the new container to it. If you want to keep the same volume, remember about removing it like so:
239 ```sh
240 docker-compose rm postgres
241 docker volume rm docker_garg-pgdata
242 ```
243
244 Now, start the container and execute:
245 ```sh
246 # need to drop the empty DB first, since schema will be created when restoring the dump
247 docker exec -i <new-container-id> dropdb -U gargantua gargandbV5
248 # recreate the db, but empty with no schema
249 docker exec -i <new-container-id> createdb -U gargantua gargandbV5
250 # now we can restore the dump
251 docker exec -i <new-container-id> psql -U gargantua -d gargandbV5 < 11-db.dump
252 ```
253
254 ### Upgrading using
255
256 There is a solution using pgupgrade_cluster but you need to manage the
257 clusters version 14 and 13. Hence here is a simple solution to upgrade.
258
259 First save your data:
260 ```
261 sudo su postgres
262 pg_dumpall > gargandb.dump
263 ```
264
265 Upgrade postgresql:
266 ```
267 sudo apt install postgresql-server-14 postgresql-client-14
268 sudo apt remove --purge postgresql-13
269 ```
270 Restore your data:
271 ```
272 sudo su postgres
273 psql < gargandb.dump
274 ```
275
276 Maybe you need to restore the gargantua password
277 ```
278 ALTER ROLE gargantua PASSWORD 'yourPasswordIn_gargantext.ini'
279 ```
280 Maybe you need to change the port to 5433 for database connection in
281 your gargantext.ini file.
282
283
284
285