Skip to content

Data import and trips#index optimizations#3

Open
holyketzer wants to merge 7 commits intospajic:masterfrom
holyketzer:master
Open

Data import and trips#index optimizations#3
holyketzer wants to merge 7 commits intospajic:masterfrom
holyketzer:master

Conversation

@holyketzer
Copy link

@holyketzer holyketzer commented Apr 5, 2019 •
edited
Loading

Время импорта файла large.json уменьшено до 12 секунд:

стало (сколько было не замерил до конца, слишком долго)

bundle exec rake "reload_json[fixtures/large.json]" 8.88s user 0.89s system 86% cpu 11.274 total

Среднее время ответа trips#index с данными из large.json сокращено с 20504 мс до 177 мс:

было:

> ab -n 10 -c 1 http://localhost:3000/автобусы/Самара/Москва

Concurrency Level:      1
Time taken for tests:   205.042 seconds
Complete requests:      10
Failed requests:        0
Total transferred:      5341362 bytes
HTML transferred:       5334370 bytes
Requests per second:    0.05 [#/sec] (mean)
Time per request:       20504.182 [ms] (mean)
Time per request:       20504.182 [ms] (mean, across all concurrent requests)
Transfer rate:          25.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing: 19314 20504 679.6  20493   21856
Waiting:    19313 20504 679.7  20493   21856
Total:      19314 20504 679.7  20493   21857

стало

> ab -n 100 -c 1 http://localhost:3000/автобусы/Самара/Москва

Concurrency Level:      1
Time taken for tests:   17.733 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      70281278 bytes
HTML transferred:       70211500 bytes
Requests per second:    5.64 [#/sec] (mean)
Time per request:       177.326 [ms] (mean)
Time per request:       177.326 [ms] (mean, across all concurrent requests)
Transfer rate:          3870.50 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   145  177  48.0    172     545
Waiting:      144  177  48.0    172     545
Total:        145  177  48.0    172     545

@holyketzer
Copy link
Author

holyketzer commented Apr 5, 2019

Это провал, я только сейчас заметил что мой фидбек-луп перетирает все данные в development соответственно я всё бенчмаркал для small.json и самое жирное не увидел. Хм ещё интересно что IPS разительно отличается для dev и тест окружения, 24.348 против 83.694.

@holyketzer
Copy link
Author

holyketzer commented Apr 5, 2019

Поправил

spajic
spajic approved these changes Apr 7, 2019
View reviewed changes
Copy link
Owner

@spajic spajic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 👍

.rspec
@@ -0,0 +1,4 @@
--require rails_helper
--profile 10
--color
Copy link
Owner

@spajic spajic Apr 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

.rspec
@@ -0,0 +1,4 @@
--require rails_helper
Copy link
Owner

@spajic spajic Apr 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

app/helpers/application_helper.rb
@@ -1,2 +1,5 @@
module ApplicationHelper
def render_delimiter
Copy link
Owner

@spajic spajic Apr 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

app/models/bus.rb
has_many :trips
has_and_belongs_to_many :services, join_table: :buses_services
has_many :bus_services
has_many :services, through: :bus_services
Copy link
Owner

@spajic spajic Apr 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

app/services/db_populator.rb
@@ -0,0 +1,52 @@
class DbPopulator
Copy link
Owner

@spajic spajic Apr 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

lib/tasks/utils.rake
)
end
end
DbPopulator.populate(args.file_name)
Copy link
Owner

@spajic spajic Apr 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@spajic spajic spajic approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@holyketzer @spajic