Yake Refactoring .1 #90

tiagolv · 2025-05-22T20:59:17Z

Refactored code, added documentation ,updated readme, better CI/CD pipelin, new webpage with documentation and new links

Teste de linting nos ficheiros atuais do yake

testes gerais

refatorização inicial do programa principal

criado workflow para verificar resultados

atualização resultados.yml

atualizado resultados.yml

atualizado status adicionados ficheiros originais para comparações

Removidos ficheiros originais, pois estavam a interferir com o score do teste pylint

highligths.py 90%

só falta refatorar variáveis, reconstruir métodos e documentação

Copilot

Pull Request Overview

This PR refactors the repository by removing legacy Docker setups and demos, and introduces improved documentation, a Makefile for common tasks, and enhanced CI/CD pipelines.

Removed outdated Dockerfiles, REST API example, and demo application
Overhauled README with new installation and usage instructions
Added Makefile and GitHub Actions workflows for install, lint, test, format, and deployment

Reviewed Changes

Copilot reviewed 155 out of 155 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
docker/Dockerfiles/yake/Dockerfile	Removed legacy YAKE CLI Dockerfile
demo/streamlit_app.py	Deleted demo Streamlit application
README.md	Revamped installation, usage, and documentation
Makefile	Added Makefile for build/test/format/lint tasks
.github/workflows/deploy.yml	New deployment workflow for GitHub Pages

Copilot · 2025-05-22T21:01:21Z

README.md

+uv pip install git+https://github.com/LIAAD/yake
 ```
-*Example text from Wikipedia*

-<a name="rest-api-image"></a>
-### Option 2. REST API Server in a Docker container
-
-This install will provide you a mirror of the original REST API of YAKE! available [here](https://boiling-castle-88317.herokuapp.com).
+or

 ```bash
-docker run -p 5000:5000 -d liaad/yake-server:latest
-```
-
-After it starts up, the container will run in the background, at http://127.0.0.1:5000. To access the YAKE! API documentation, go to http://127.0.0.1:5000/apidocs/.
-
-You can test the RESTful API using `curl`:
-
-```bash
-curl -X POST "http://localhost:5000/yake/" -H "accept: application/json" -H "Content-Type: application/json" \
-d @- <<'EOF'
-{
-  "language": "en",
-  "max_ngram_size": 3,
-  "number_of_keywords": 10,
-  "text": "Sources tell us that Google is acquiring Kaggle, a platform that hosts data science and machine learning competitions. Details about the transaction remain somewhat vague , but given that Google is hosting its Cloud Next conference in San Francisco this week, the official announcement could come as early as tomorrow. Reached by phone, Kaggle co-founder CEO Anthony Goldbloom declined to deny that the acquisition is happening. Google itself declined 'to comment on rumors'. Kaggle, which has about half a million data scientists on its platform, was founded by Goldbloom and Ben Hamner in 2010. The service got an early start and even though it has a few competitors like DrivenData, TopCoder and HackerRank, it has managed to stay well ahead of them by focusing on its specific niche. The service is basically the de facto home for running data science and machine learning competitions. With Kaggle, Google is buying one of the largest and most active communities for data scientists ..."
-}
-EOF
-```
-*Example text from Wikipedia*
-
-<a name="standalone-installation"></a>
-### Option 3. Standalone Installation (for development or integration)
-
-#### Requirements
-
-Python3
-
-#### Installation
-
-To install Yake using pip:
-
-``` bash
-pip install git+https://github.com/LIAAD/yake
+uv pip install yake


The command prefix uv is unclear. If this is not an alias defined elsewhere, consider replacing it with a standard pip install invocation or documenting the alias.

Copilot · 2025-05-22T21:01:21Z

README.md

+uv pip install git+https://github.com/LIAAD/yake
 ```
-*Example text from Wikipedia*

-<a name="rest-api-image"></a>
-### Option 2. REST API Server in a Docker container
-
-This install will provide you a mirror of the original REST API of YAKE! available [here](https://boiling-castle-88317.herokuapp.com).
+or

 ```bash
-docker run -p 5000:5000 -d liaad/yake-server:latest
-```
-
-After it starts up, the container will run in the background, at http://127.0.0.1:5000. To access the YAKE! API documentation, go to http://127.0.0.1:5000/apidocs/.
-
-You can test the RESTful API using `curl`:
-
-```bash
-curl -X POST "http://localhost:5000/yake/" -H "accept: application/json" -H "Content-Type: application/json" \
-d @- <<'EOF'
-{
-  "language": "en",
-  "max_ngram_size": 3,
-  "number_of_keywords": 10,
-  "text": "Sources tell us that Google is acquiring Kaggle, a platform that hosts data science and machine learning competitions. Details about the transaction remain somewhat vague , but given that Google is hosting its Cloud Next conference in San Francisco this week, the official announcement could come as early as tomorrow. Reached by phone, Kaggle co-founder CEO Anthony Goldbloom declined to deny that the acquisition is happening. Google itself declined 'to comment on rumors'. Kaggle, which has about half a million data scientists on its platform, was founded by Goldbloom and Ben Hamner in 2010. The service got an early start and even though it has a few competitors like DrivenData, TopCoder and HackerRank, it has managed to stay well ahead of them by focusing on its specific niche. The service is basically the de facto home for running data science and machine learning competitions. With Kaggle, Google is buying one of the largest and most active communities for data scientists ..."
-}
-EOF
-```
-*Example text from Wikipedia*
-
-<a name="standalone-installation"></a>
-### Option 3. Standalone Installation (for development or integration)
-
-#### Requirements
-
-Python3
-
-#### Installation
-
-To install Yake using pip:
-
-``` bash
-pip install git+https://github.com/LIAAD/yake
+uv pip install yake


Similar to above, the uv prefix is ambiguous to users. Update to pip install yake or explain what uv refers to in this context.

Copilot · 2025-05-22T21:01:21Z

README.md

+	  --help                          Show this message and exit.
+```
+
+Dont know which Deduplication function to use, see more [here](https://tiagolv.github.io/yakerf/docs/-getting-started#keyword-deduplication-methods)


Typo: Dont should be Don't (with an apostrophe).

Suggested change

Dont know which Deduplication function to use, see more [here](https://tiagolv.github.io/yakerf/docs/-getting-started#keyword-deduplication-methods)

Don't know which Deduplication function to use, see more [here](https://tiagolv.github.io/yakerf/docs/-getting-started#keyword-deduplication-methods)

Copilot · 2025-05-22T21:01:22Z

.github/workflows/deploy.yml

+on:
+  push:
+    branches:
+      - core-seperation  # ou master, dependendo da sua branch principal


Misspelling: core-seperation should be core-separation.

Suggested change

- core-seperation # ou master, dependendo da sua branch principal

- core-separation # ou master, dependendo da sua branch principal

Copilot · 2025-05-22T21:01:22Z

Makefile

+	uv pip install --upgrade pip
+	uv pip install -e .
+
+install-dev:
+	uv pip install --upgrade pip
+	uv pip install -e ".[dev]"
+
+test:
+	uv run pytest -vv --cov=yake test_*.py
+
+format:
+	uv run black .
+
+lint:
+	uv run ruff check --fix .
+	uv run ruff check .
+	uv run flake8 yake/


The uv prefix is not a standard command; consider using python -m pip install --upgrade pip or explain uv if it's an alias.

Suggested change

uv pip install --upgrade pip

uv pip install -e .

install-dev:

uv pip install --upgrade pip

uv pip install -e ".[dev]"

test:

uv run pytest -vv --cov=yake test_*.py

format:

uv run black .

lint:

uv run ruff check --fix .

uv run ruff check .

uv run flake8 yake/

python -m pip install --upgrade pip

python -m pip install -e .

install-dev:

python -m pip install --upgrade pip

python -m pip install -e ".[dev]"

test:

pytest -vv --cov=yake test_*.py

format:

black .

lint:

ruff check --fix .

ruff check .

flake8 yake/

Copilot · 2025-05-22T21:01:22Z

.github/workflows/publishpypi.yml

+    name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@master


Referencing the master branch in an action version is unstable. It’s recommended to pin to a specific version or tag (e.g., actions/checkout@v3).

Suggested change

- uses: actions/checkout@master

- uses: actions/checkout@v3

tiagolv added 30 commits November 6, 2024 13:19

Create super-linter.yml

5f28658

Teste de linting nos ficheiros atuais do yake

Create pylint.yml

0e13bc5

testes gerais

atualização das imagens das dockers

ceef896

yake.py 60%

e35fb82

refatorização inicial do programa principal

Create resultados.yml

f678242

criado workflow para verificar resultados

Update resultados.yml

729d101

atualização resultados.yml

Update resultados.yml

de7bd0f

atualizado resultados.yml

pyint espaços + atualização de status

d179ba3

link code testes

522d95a

Levenshtein refatorização inicial

bb9c340

clip.py - initial refactoring

79315a7

atualizado status adicionados ficheiros originais para comparações

removidos ficheiros originais

ecddac4

Removidos ficheiros originais, pois estavam a interferir com o score do teste pylint

cli.py atualizado

3d7c317

trailing whitespaces

956bb97

highlight refatorização inicial

cf3ef99

highlight.py lines

2aadc64

highlight.py docstrings

56b48b2

datarepresentation.py formatação inicial

9a13e29

highligths.py 90%

datarepresentatio 40%

d4cfd88

datarepresentation linting

8755e64

datarepresentation 70%

df183c7

só falta refatorar variáveis, reconstruir métodos e documentação

variaveis

cf59a6b

+ variables

b8cdb5c

25/02/2025 acabado

f8672e3

teste yake.py métodos

f4bd1ed

yake.py métodos e atributos distribuidos

0a3a482

yake.py 100% Refactored and optimized

116a82e

separação de métodos highlight.py

36b4a08

documentação beta highlight.py

33d692f

+ documentação

b25cc10

tiagolv added 25 commits May 20, 2025 02:37

final docs website structure

963687a

Update about.mdx

cd1b375

Update about.mdx

d1800c8

compatibility error

b3fd211

index

00230a2

updated formattting

0ada57f

icon test

7c11fb4

docs

8ce7ca4

added notebook

fb72306

updated collab redirections and notebook

921fbd3

Create meta.json

44446e9

updated order

d1fef97

Delete meta.json

7343b3e

Update getting-started.mdx

fb3a3cf

Update README.md

ab73131

sidebar test

75a81d7

teste sidebar

b0fd520

_meta

ce9fc5f

updated sidebar

d538819

sidebar final config

9b99efe

updated utils and homepage

b96f10a

Update README.md

31d32e6

updated main class documentation

807453a

changed to uv

dac7781

finishing touches

a4f7027

tiagolv requested review from Copilot and rncampos May 22, 2025 20:59

Copilot AI reviewed May 22, 2025

View reviewed changes

updated workflows for uv

2e2fa6c

tiagolv closed this May 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Yake Refactoring .1 #90

Yake Refactoring .1 #90

Uh oh!

tiagolv commented May 22, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	Dont know which Deduplication function to use, see more [here](https://tiagolv.github.io/yakerf/docs/-getting-started#keyword-deduplication-methods)
	Don't know which Deduplication function to use, see more [here](https://tiagolv.github.io/yakerf/docs/-getting-started#keyword-deduplication-methods)

	- core-seperation # ou master, dependendo da sua branch principal
	- core-separation # ou master, dependendo da sua branch principal

Yake Refactoring .1 #90

Yake Refactoring .1 #90

Uh oh!

Conversation

tiagolv commented May 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant