Skip to content

upgrade to Ruby 3, update linux demo docs + development dependency #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
almereyda wants to merge 3 commits into
base: master
Choose a base branch
from
Open

upgrade to Ruby 3, update linux demo docs + development dependency #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5d91442
upgrade server to ruby 3 + add development dependency
almereyda Apr 25, 2025
9d6432c
add updating steps to linux demo
almereyda Apr 25, 2025
5ced454
refresh linux demo example and fix mistakes
almereyda Apr 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,5 @@ source "https://rubygems.org"
gem "sinatra", "~> 4.1"
gem "rackup", "~> 2.2"
gem "puma", "~> 6.6"

gem "solargraph", "~> 0.54.0", group: :development
96 changes: 94 additions & 2 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,49 @@
GEM
remote: https://rubygems.org/
specs:
ast (2.4.3)
backport (1.2.0)
base64 (0.2.0)
benchmark (0.4.0)
diff-lcs (1.6.1)
jaro_winkler (1.6.0)
json (2.11.3)
kramdown (2.5.1)
rexml (>= 3.3.9)
kramdown-parser-gfm (1.1.0)
kramdown (~> 2.0)
language_server-protocol (3.17.0.4)
lint_roller (1.1.0)
logger (1.7.0)
mustermann (3.0.3)
ruby2_keywords (~> 0.0.1)
nio4r (2.7.4)
nokogiri (1.18.8-aarch64-linux-gnu)
racc (~> 1.4)
nokogiri (1.18.8-aarch64-linux-musl)
racc (~> 1.4)
nokogiri (1.18.8-arm-linux-gnu)
racc (~> 1.4)
nokogiri (1.18.8-arm-linux-musl)
racc (~> 1.4)
nokogiri (1.18.8-arm64-darwin)
racc (~> 1.4)
nokogiri (1.18.8-x86_64-darwin)
racc (~> 1.4)
nokogiri (1.18.8-x86_64-linux-gnu)
racc (~> 1.4)
nokogiri (1.18.8-x86_64-linux-musl)
racc (~> 1.4)
observer (0.1.2)
ostruct (0.6.1)
parallel (1.27.0)
parser (3.3.8.0)
ast (~> 2.4.1)
racc
prism (1.4.0)
puma (6.6.0)
nio4r (~> 2.0)
racc (1.8.1)
rack (3.1.13)
rack-protection (4.1.1)
base64 (>= 0.1.0)
Expand All @@ -18,6 +54,28 @@ GEM
rack (>= 3.0.0)
rackup (2.2.1)
rack (>= 3)
rainbow (3.1.1)
rbs (3.9.2)
logger
regexp_parser (2.10.0)
reverse_markdown (3.0.0)
nokogiri
rexml (3.4.1)
rubocop (1.75.3)
json (~> 2.3)
language_server-protocol (~> 3.17.0.2)
lint_roller (~> 1.1.0)
parallel (~> 1.10)
parser (>= 3.3.0.2)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 2.9.3, < 3.0)
rubocop-ast (>= 1.44.0, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 4.0)
rubocop-ast (1.44.1)
parser (>= 3.3.7.2)
prism (~> 1.4)
ruby-progressbar (1.13.0)
ruby2_keywords (0.0.5)
sinatra (4.1.1)
logger (>= 1.6.0)
Expand All @@ -26,15 +84,49 @@ GEM
rack-protection (= 4.1.1)
rack-session (>= 2.0.0, < 3)
tilt (~> 2.0)
solargraph (0.54.0)
backport (~> 1.2)
benchmark
bundler (~> 2.0)
diff-lcs (~> 1.4)
jaro_winkler (~> 1.6)
kramdown (~> 2.3)
kramdown-parser-gfm (~> 1.1)
logger (~> 1.6)
observer (~> 0.1)
ostruct (~> 0.6)
parser (~> 3.0)
rbs (~> 3.3)
reverse_markdown (>= 2.0, < 4)
rubocop (~> 1.38)
thor (~> 1.0)
tilt (~> 2.0)
yard (~> 0.9, >= 0.9.24)
yard-solargraph (~> 0.1)
thor (1.3.2)
tilt (2.6.0)
unicode-display_width (3.1.4)
unicode-emoji (~> 4.0, >= 4.0.4)
unicode-emoji (4.0.4)
yard (0.9.37)
yard-solargraph (0.1.0)
yard (~> 0.9)

PLATFORMS
aarch64-linux-gnu
aarch64-linux-musl
arm-linux-gnu
arm-linux-musl
arm64-darwin
x86_64-darwin
x86_64-linux-gnu
x86_64-linux-musl

DEPENDENCIES
puma (~> 6.6)
rackup (~> 2.2)
sinatra (= 4.1.1)
sinatra (~> 4.1)
solargraph (~> 0.54.0)

BUNDLED WITH
2.4.20
2.6.8
102 changes: 78 additions & 24 deletions demo/linux.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,17 @@ export PATH="$GEM_HOME/bin:$PATH"
Install the Ruby dependencies with:

```sh
bundle install
bundle install --without development
```

This spares you from installing the `solargraph` development dependency for the
Ruby language server into your demo system.

## Preparation

The code assumes in many place from where it is made available.
Adapt this location to your preference.
The code references the assumed public location of its endpoint and the endpoint
of the associated search wiki in many places. Adapt this location to your
preference.

```sh
xargs -L1 sed 's/search.fed.wiki.org/search.federatedwiki.org/g' -i <<<"online.pl
Expand All @@ -49,13 +53,6 @@ xargs -L1 sed 's/search.fed.wiki.org/search.federatedwiki.org/g' -i <<<"online.p
server.rb"
```

We also need a little upgrade, if we want to run from Ruby 3.

```sh
xargs -L1 sed 's/Dir.exists/Dir.exist/' -i <<<"slug-web.rb
site-web.rb"
```

You will need at least one seed site to start from. Additionally we prepare the
retired store.

Expand All @@ -68,13 +65,17 @@ LANG="en_US.UTF-8" sh cron.sh

Initially, this will take a lot of time. Wait for the process to complete.

Observe its doings for a current time window with e.g.:
Observe its doings for a current interval with e.g.:

```sh
tail -f logs/Tue-1800
```

An exemplary initial run took four and a half hours.
An exemplary initial run took four and a half hours. It took ten runs of the
scheduled crawling to discover a stable set of federated domains.

This allows [the logs](http://search.federatedwiki.org:3030/logs) to be used as
an activity monitor for each interval they cover.

## Running

Expand Down Expand Up @@ -131,30 +132,40 @@ systemctl enable --user --now wiki-search

The search is now available at <http://search.federatedwiki.org:3030>.

Persist the user service with enabling linger mode:
Persist the user service across reboots with enabling user lingering:

```sh
sudo loginctl enable-linger ubuntu
```

### Proxy
### HTTP & HTTPS/ACME proxy Caddy

For serving the query interface using the default HTTP port, use:
For serving the query interface using the default HTTP port and separately via
HTTPS without issuing a redirect, you can use the caddy HTTP server.

```
Modify the email address and domain to match your environment.

```sh
sudo cat <<CADDYFILE > /etc/caddy/Caddyfile
> {
{
email acme@search.federatedwiki.org
}

http://query.search.federatedwiki.org, https://query.search.federatedwiki.org {
reverse_proxy localhost:3030
}
CADDYFILE
sudo systemctl restart caddy
```

The query interface for the search is then also available at <http://query.search.federatedwiki.org>.
```sh
sudo systemctl reload caddy
```

The query interface for the search is then also available at
<http://query.search.federatedwiki.org> and
<https://query.search.federatedwiki.org>.

The latter offers HTTPS-enabled wikis to use the search.

### Wiki

Expand All @@ -166,7 +177,7 @@ This needs an additional installation of

You can grab one at <https://nvm.sh>.

Copy the configuration example in place and fill the secret value.
Copy the configuration example and fill the secret value.

```sh
cp config.json.example config.json
Expand All @@ -176,15 +187,15 @@ cp config.json.example config.json

```sh
sed 's/ROSTER search.fed.wiki.org:3030/ROSTER query.search.federatedwiki.org/' -i pages/federation-search
sed 's/ROSTER search.fed.wiki.org/ROSTER query.search.federatedwiki.org/' -i pages/federation-search
sed 's/ROSTER search.fed.wiki.org/ROSTER search.federatedwiki.org/' -i pages/federation-search
xargs -L1 sed 's|http://search.fed.wiki.org:3030|http://query.search.federatedwiki.org|g' -i <<<"pages/federation-search
pages/search-help"
sed 's/"search.fed.wiki.org:3030"/"query.search.federatedwiki.org"/g' -i pages/federation-search
```

</details>

Now you can try to run the wiki.
You can now try to run the wiki.

```
wiki --data "${PWD}" --config "${PWD}/config.json"
Expand All @@ -193,6 +204,8 @@ wiki --data "${PWD}" --config "${PWD}/config.json"
If everything looks good when visiting <http://search.federatedwiki.org:3000>,
you can add the service to your service manager and reverse proxy.

### Persistent system service

```sh
systemctl edit --user --force --full wiki
```
Expand Down Expand Up @@ -220,15 +233,17 @@ WantedBy=default.target
Tell your service manager about it:

```sh
systemctl --user daemon-reload
systemctl daemon-reload --user
```

And enable it:

```sh
systemctl --user enable --now wiki
systemctl enable --user --now wiki
```

### Proxy configuration

Next add the service to the proxy:

```sh
Expand All @@ -245,3 +260,42 @@ systemctl reload caddy
```

The wiki is now available at <http://search.federatedwiki.org/>.

## Updating

To update your search service, stop its runtime, keep your modifications,
pull available changes, reapply your changes and boot up again.

We'll walk you through the steps.

1. Stop services

```sh
systemctl stop --user wiki-search wiki
```

2. Keep local motifications

```sh
git stash push
```

3. Pull remote motifications

```sh
git pull
```

4. Restore local motifications

```sh
git stash pop
```

5. Restart services

```sh
systemctl start --user wiki-search wiki
```

You're now up to date and benefitting from latest changes.
2 changes: 1 addition & 1 deletion site-web.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
def read
Dir.new('sites').each do |site|
next if site.match /^\./
next unless Dir.exists?("sites/#{site}/pages")
next unless Dir.exist?("sites/#{site}/pages")
next if (Date.today - File.mtime("sites/#{site}/pages").to_date).to_i > 365
size = Dir["sites/#{site}/pages/*"].size
next if size < 10
Expand Down
4 changes: 2 additions & 2 deletions slug-web.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
def read
Dir.new('sites').each do |site|
next if site.match /^\./
next unless Dir.exists?("sites/#{site}/pages")
next unless Dir.exist?("sites/#{site}/pages")
Dir["sites/#{site}/pages/*"].each do |path|
slug = (path.chomp.split(/\//))[3]
@web[slug][:forks] += 1
Expand All @@ -25,4 +25,4 @@ def clean
read
clean

puts JSON.pretty_generate @web
puts JSON.pretty_generate @web