OOP school library: add basic UI

[ ] Create a app.rb file that will serve as your console app entry-point. It should have methods that do the following:
[ ] List all books.
[ ] List all people.
[ ] Create a person (teacher or student, not a plain Person).
[ ] CreaCreate a app.rb file that will serve as your console app entry-point. It should have methods that do the following:
[ ] List all books.
[ ] List all people.
[ ] Create a person (teacher or student, not a plain Person).
[ ] Create a book.
[ ] Create a rental.
[ ] List all rentals for a given person id.
[ ] In your main.rb define the entry point, this will be a method called main that is invoked at the end of your file. This method should do the following:
[ ] Present the user with a list of options to perform.
[ ] Lets users choose an option.
[ ] If needed, ask for parameters for the option.
[ ] Have a way to quit the app.te a book.
[ ] Create a rental.
[ ] List all rentals for a given person id.
[ ] In your main.rb define the entry point, this will be a method called main that is invoked at the end of your file. This method should do the following:
[ ] Present the user with a list of options to perform.
[ ] Lets users choose an option.
[ ] If needed, ask for parameters for the option.
[ ] Have a way to quit the app.

Hi @rloterh kindly can you be clear with the changes you want me to make, when you just say not checked what do you me. elaborate

ok


devices: inform user that tensorflow wasn't compiled with CUDA support

The current behaviour can be confusing when no GPU are found on a computer that does have GPUs, so add a warning.

Nagging Reviewer @hawkinsp: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 14 days with no activity and the awaiting review label has been applied.


172 | Factorial Trailing Zeroes | Python

Factorial Trailing Zeroes
What will change
Type of Issue

[x] Adding New Code

Programming Language

[x] Python

Self Check

Ask for issue assignment and wait to get assigned to it before making Pull Request.
Add your file in the proper folder
Clean Code and Documentation for better readability
Add Title and Description of the program in the file

:star2: Star it :fork_and_knife:Fork it :handshake: Contribute to it!
Discord server ✉ - https://discord.gg/CXksGdn
Happy Coding,

I'd like to get assigned


Review SQL in TTAPI course search endpoint

Context
The endpoint that Find uses for course searches is high traffic and can be slow for certain queries. Therefore we have attempted to analyse the database queries to get a handle on what they are doing, how slow or fast they are and come up with any optimisations that are possible.
https://trello.com/c/7ps7weg2/3953-review-sql-in-ttapi-course-search-endpoint
Analysis
Find has many options for course searches. First the candidate selects one of three basic options:

If location is selected the candidate provides a location and the search results are ordered by distance from the at location. If a provider name is given then the search is filtered by provider, otherwise if the Across England option is selected no location based filter is applied.
Next the candidate picks one or more course subjects:

Then from the results page additional filters can be applied:

Example 1
The first example is a fairly simple query Across England with a few filters selected, Find path is /results?subjects%5B%5D=31&hasvacancies=false&hasvacancies=true&fulltime=true&qualifications%5B%5D=QtsOnly&funding=8&l=2
The log records a number of SQL statements:
{"name":"Rails","message":"API HIT =\u003e http://localhost:3001/api/v3/recruitment_cycles/2021/courses?filter%5Bfunding%5D=salary\u0026filter%5Bhas_vacancies%5D=true\u0026filter%5Bqualification%5D=qts\u0026filter%5Bstudy_type%5D=full_time\u0026filter%5Bsubjects%5D=00\u0026include=site_statuses.site%2Cprovider%2Csubjects\u0026page%5Bpage%5D=1\u0026page%5Bper_page%5D=10\u0026sort=provider.provider_name%2Cname"}
{"name":"Rack","message":"Started","payload":{"method":"GET","path":"/api/v3/recruitment_cycles/2021/courses?filter%5Bfunding%5D=salary\u0026filter%5Bhas_vacancies%5D=true\u0026filter%5Bqualification%5D=qts\u0026filter%5Bstudy_type%5D=full_time\u0026filter%5Bsubjects%5D=00\u0026include=site_statuses.site%2Cprovider%2Csubjects\u0026page%5Bpage%5D=1\u0026page%5Bper_page%5D=10\u0026sort=provider.provider_name%2Cname","ip":"::1"}}
{"name":"API::V3::CoursesController","message":"Processing #index"}
SELECT \"recruitment_cycle\".*   FROM \"course_enrichment\" WHERE \"course_enrichment\".\"course_id\" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)","binds":{"course_id":[12954225,12944992,12941095,12941113,12942478,12938281,12953561,12944047,12938982,12938743]},"allocations":766,"cached":null}}
SELECT \"course_subject\".* FROM \"course_subject\" WHERE \"course_subject\".\"course_id\" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) ORDER BY \"course_subject\".\"position\" ASC","binds":{"course_id":[12954225,12944992,12941095,12941113,12942478,12938281,12953561,12944047,12938982,12938743]},"allocations":322,"cached":null}}
SELECT \"subject\".* FROM \"subject\" WHERE \"subject\".\"id\" = $1","binds":{"id":1},"allocations":686,"cached":null}}
SELECT \"financial_incentive\".* FROM \"financial_incentive\" WHERE \"financial_incentive\".\"subject_id\" = $1","binds":{"subject_id":1},"allocations":196,"cached":null}}
SELECT \"course_site\".* FROM \"course_site\" WHERE \"course_site\".\"course_id\" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)","binds":{"course_id":[12954225,12944992,12941095,12941113,12942478,12938281,12953561,12944047,12938982,12938743]},"allocations":367,"cached":null}}
SELECT \"site\".* FROM \"site\" WHERE \"site\".\"id\" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24)","binds":{"id":[11219340,11219487,11219607,11219605,11219608,11219611,11219610,11219609,11219606,11219604,11219601,11220604,11221198,11221905,11221915,11221904,11221909,11221893,11221895,11221900,11221891,11222303,11225973,11226334]},"allocations":675,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9)","binds":{"id":[16984,16315,16028,16122,15810,16931,16240,15863,15843]},"allocations":700,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1","binds":{"id":3},"allocations":309,"cached":null}}
SELECT \"provider_ucas_preference\".*  FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"T89","recruitment_cycle_id":3,"limit":1},"allocations":349,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"1WR","recruitment_cycle_id":3,"limit":1},"allocations":408,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"2B5","recruitment_cycle_id":3,"limit":1},"allocations":440,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"2B5","recruitment_cycle_id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"1WR","recruitment_cycle_id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"M80","recruitment_cycle_id":3,"limit":1},"allocations":440,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"P63","recruitment_cycle_id":3,"limit":1},"allocations":439,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"C10","recruitment_cycle_id":3,"limit":1},"allocations":439,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"P85","recruitment_cycle_id":3,"limit":1},"allocations":626,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"provider_code\" = $1 AND \"provider\".\"recruitment_cycle_id\" = $2 LIMIT $3","binds":{"provider_code":"24S","recruitment_cycle_id":3,"limit":1},"allocations":254,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":16984,"recruitment_cycle_id":3},"allocations":1490,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":16315,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":16028,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":16122,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":15810,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":16931,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":16240,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":15863,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT DISTINCT \"provider\".* FROM \"provider\" INNER JOIN \"course\" ON \"provider\".\"provider_code\" = \"course\".\"accredited_body_code\" WHERE \"course\".\"provider_id\" = $1 AND \"course\".\"discarded_at\" IS NULL AND \"provider\".\"recruitment_cycle_id\" = $2","binds":{"provider_id":15843,"recruitment_cycle_id":3},"allocations":401,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16984,"limit":1},"allocations":8,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":225,"cached":null}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16315,"limit":1},"allocations":401,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16028,"limit":1},"allocations":401,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16122,"limit":1},"allocations":400,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15810,"limit":1},"allocations":401,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16931,"limit":1},"allocations":400,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":401,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":16240,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":401,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15863,"limit":1},"allocations":1,"cached":true}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
SELECT \"provider\".* FROM \"provider\" WHERE \"provider\".\"id\" = $1 LIMIT $2","binds":{"id":15843,"limit":1},"allocations":402,"cached":null}}
SELECT \"recruitment_cycle\".* FROM \"recruitment_cycle\" WHERE \"recruitment_cycle\".\"id\" = $1 LIMIT $2","binds":{"id":3,"limit":1},"allocations":1,"cached":true}}
{"name":"Rails","message":"Completed JSON API rendering (302.45ms)"}
{"duration_ms":640.4970000148751,"duration":"640.5ms","name":"API::V3::CoursesController","message":"Completed #index","payload":{"controller":"API::V3::CoursesController","action":"index","params":{"filter":{"funding":"salary","has_vacancies":"true","qualification":"qts","study_type":"full_time","subjects":"00"},"include":"site_statuses.site,provider,subjects","page":{"page":"1","per_page":"10"},"sort":"provider.provider_name,name","recruitment_cycle_year":"2021"},"format":"JSONAPI","method":"GET","path":"/api/v3/recruitment_cycles/2021/courses","status":200,"view_runtime":171.79,"db_runtime":457.26,"allocations":93769,"status_message":"OK"}}

There is a lot happening here, including some potential 1+n queries, however the bulk of the time is spent in a couple of queries like this (the other one does a count):
SELECT provider.provider_name, course.*
FROM course
INNER JOIN provider ON provider.id = course.provider_id
WHERE course.id IN (
  SELECT course.id
  FROM course
  INNER JOIN provider ON course.provider_id = provider.id
  INNER JOIN course_site ON course_site.course_id = course.id
  INNER JOIN course_subject ON course_subject.course_id = course.id
  INNER JOIN subject ON subject.id = course_subject.subject_id
  WHERE provider.recruitment_cycle_id = 3
    AND provider.discarded_at IS NULL
    AND course.discarded_at IS NULL
    AND course.program_type IN ('SS', 'TA')
    AND course.qualification = 0
    AND course_site.vac_status != ''
    AND course_site.status = 'R'
    AND course_site.publish = 'Y'
    AND course.study_mode IN ('F', 'B')
    AND subject.subject_code = '00'
  ) ORDER BY provider.provider_name ASC, name asc, course_code asc LIMIT 10 OFFSET 0;

This is basically a query on the course table with joins to provider, course_site, course_subject and subject. It's made a little more complicated by the filters being applied in a nested query, with the outer query just providing the SELECT clause and handling pagination and sorting.
EXPLAIN (ANALYSE, BUFFERS) gives the following breakdown:
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                QUERY PLAN                                                                 │
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Limit  (cost=1988.96..1988.98 rows=8 width=1173) (actual time=69.144..69.152 rows=10 loops=1)                                             │
│   Buffers: shared hit=47763                                                                                │
│   ->  Sort  (cost=1988.96..1988.98 rows=8 width=1173) (actual time=69.143..69.149 rows=10 loops=1)                                        │
│         Sort Key: provider.provider_name, course.name, course.course_code                                                                 │
│         Sort Method: top-N heapsort  Memory: 28kB                                                                                │
│         Buffers: shared hit=47763                                                                                │
│         ->  Nested Loop  (cost=1955.81..1988.84 rows=8 width=1173) (actual time=66.931..67.393 rows=69 loops=1)                           │
│               Buffers: shared hit=47763                                                                                │
│               ->  Nested Loop  (cost=1955.53..1985.31 rows=8 width=1144) (actual time=66.894..67.186 rows=69 loops=1)                     │
│                     Buffers: shared hit=47556                                                                                │
│                     ->  HashAggregate  (cost=1955.11..1955.19 rows=8 width=12) (actual time=66.198..66.225 rows=69 loops=1)               │
│                           Group Key: course_1.id                                                                                │
│                           Batches: 1  Memory Usage: 32kB                                                                                │
│                           Buffers: shared hit=47280                                                                                │
│                           ->  Nested Loop  (cost=27.53..1955.09 rows=8 width=12) (actual time=26.407..65.451 rows=125 loops=1)            │
│                                 Buffers: shared hit=47280                                                                                │
│                                 ->  Nested Loop  (cost=27.25..1874.99 rows=30 width=16) (actual time=1.961..44.828 rows=176 loops=1)      │
│                                       Join Filter: (course_site.course_id = course_1.id)                                                  │
│                                       Buffers: shared hit=46752                                                                           │
│                                       ->  Nested Loop  (cost=26.83..1422.11 rows=640 width=8) (actual time=1.311..33.837 rows=3383 loops=…│
│…1)                                                                                │
│                                             Buffers: shared hit=33220                                                                     │
│                                             ->  Nested Loop  (cost=26.54..632.45 rows=1838 width=4) (actual time=1.253..7.321 rows=10655 …│
│…loops=1)                                                                                │
│                                                   Buffers: shared hit=566                                                                 │
│                                                   ->  Seq Scan on subject  (cost=0.00..1.56 rows=1 width=8) (actual time=0.017..0.024 row…│
│…s=1 loops=1)                                                                                │
│                                                         Filter: (subject_code = '00'::text)                                               │
│                                                         Rows Removed by Filter: 44                                                        │
│                                                         Buffers: shared hit=1                                                             │
│                                                   ->  Bitmap Heap Scan on course_subject  (cost=26.54..612.51 rows=1838 width=8) (actual …│
│…time=1.230..5.420 rows=10655 loops=1)                                                                                │
│                                                         Recheck Cond: (subject_id = subject.id)                                           │
│                                                         Heap Blocks: exact=555                                                            │
│                                                         Buffers: shared hit=565                                                           │
│                                                         ->  Bitmap Index Scan on index_course_subject_on_subject_id  (cost=0.00..26.08 ro…│
│…ws=1838 width=0) (actual time=1.158..1.158 rows=10655 loops=1)                                                                            │
│                                                               Index Cond: (subject_id = subject.id)                                       │
│                                                               Buffers: shared hit=10                                                      │
│                                             ->  Index Scan using "IX_course_site_course_id" on course_site  (cost=0.29..0.42 rows=1 width…│
│…=4) (actual time=0.002..0.002 rows=0 loops=10655)                                                                                │
│                                                   Index Cond: (course_id = course_subject.course_id)                                      │
│                                                   Filter: ((vac_status <> ''::text) AND (status = 'R'::text) AND (publish = 'Y'::text))   │
│                                                   Rows Removed by Filter: 1                                                               │
│                                                   Buffers: shared hit=32654                                                               │
│                                       ->  Index Scan using "PK_course" on course course_1  (cost=0.42..0.70 rows=1 width=8) (actual time=…│
│…0.003..0.003 rows=0 loops=3383)                                                                                │
│                                             Index Cond: (id = course_subject.course_id)                                                   │
│                                             Filter: ((discarded_at IS NULL) AND (program_type = ANY ('{SS,TA}'::text[])) AND (study_mode …│
│…= ANY ('{F,B}'::text[])) AND (qualification = 0))                                                                                │
│                                             Rows Removed by Filter: 1                                                                     │
│                                             Buffers: shared hit=13532                                                                     │
│                                 ->  Index Scan using "PK_ucas_provider" on provider provider_1  (cost=0.28..2.67 rows=1 width=4) (actual …│
│…time=0.116..0.116 rows=1 loops=176)                                                                                │
│                                       Index Cond: (id = course_1.provider_id)                                                             │
│                                       Filter: ((discarded_at IS NULL) AND (recruitment_cycle_id = 3))                                     │
│                                       Rows Removed by Filter: 0                                                                           │
│                                       Buffers: shared hit=528                                                                             │
│                     ->  Index Scan using "PK_course" on course  (cost=0.42..3.77 rows=1 width=1144) (actual time=0.013..0.013 rows=1 loop…│
│…s=69)                                                                                │
│                           Index Cond: (id = course_site.course_id)                                                                        │
│                           Buffers: shared hit=276                                                                                │
│               ->  Index Scan using "PK_ucas_provider" on provider  (cost=0.28..0.44 rows=1 width=33) (actual time=0.002..0.002 rows=1 loo…│
│…ps=69)                                                                                │
│                     Index Cond: (id = course.provider_id)                                                                                │
│                     Buffers: shared hit=207                                                                                │
│ Planning:                                                                                │
│   Buffers: shared hit=132                                                                                │
│ Planning Time: 17.067 ms                                                                                │
│ Execution Time: 69.672 ms                                                                                │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
(59 rows)

Time: 123.671 ms

My reading of this is that the most expensive parts of the query are joins, specifically the two inner most nested loops and associated filters.
Example 2
The second example is a search by location /results?c=England&fulltime=true&hasvacancies=true&l=1&lat=51.588868&lng=-1.426453&loc=Wantage+OX12%2C+UK&lq=Wantage&parttime=false&qualifications%5B%5D=PgdePgceWithQts&qualifications%5B%5D=Other&rad=50&sortby=2&subjects%5B%5D=31
This one generates a more complicated main query:
SELECT course.*,
  distance,
  (CASE WHEN provider.provider_type = 'O' THEN (distance - 10) ELSE distance END) as boosted_distance
FROM course
  INNER JOIN (
    SELECT course_id, MIN(ACOS(least(1,COS(0.900395603976742)*COS(-0.02489630147495067)*COS(RADIANS(site.latitude))*COS(RADIANS(site.longitude))+COS(0.900395603976742)*SIN(-0.02489630147495067)*COS(RADIANS(site.latitude))*SIN(RADIANS(site.longitude))+SIN(0.900395603976742)*SIN(RADIANS(site.latitude))))*3963.1899999999996) as distance
    FROM course_site
      INNER JOIN site ON course_site.site_id = site.id
    WHERE (course_site.status = 'R'
      AND course_site.publish = 'Y'
      AND course_site.vac_status = 'F'
      OR course_site.status = 'R'
      AND course_site.publish = 'Y'
      AND course_site.vac_status = 'P'
      OR course_site.status = 'R'
      AND course_site.publish = 'Y'
      AND course_site.vac_status = 'B'
    ) AND site.latitude IS NOT NULL
      AND site.longitude IS NOT NULL
      AND (site.address1 != '' OR site.postcode != '')
    GROUP BY course_id) distances ON course.id = distances.course_id
  INNER JOIN provider ON provider.id = course.provider_id
  WHERE course.id IN F
    SELECT course.id FROM course
    INNER JOIN provider ON course.provider_id = provider.id
    INNER JOIN course_site ON course_site.course_id = course.id
    INNER JOIN course_subject ON course_subject.course_id = course.id
    INNER JOIN subject ON subject.id = course_subject.subject_id
    INNER JOIN course_site site_statuses_course_join ON site_statuses_course_join.course_id = course.id
    INNER JOIN site ON site.id = site_statuses_course_join.site_id AND course_site.status IN ('N', 'R')
    WHERE provider.recruitment_cycle_id = 3
      AND provider.discarded_at IS NULL
      AND course.discarded_at IS NULL
      AND course.qualification IN (1, 2, 3, 4)
      AND course_site.vac_status != ''
      AND course_site.status = 'R'
      AND course_site.publish = 'Y'
      AND course.study_mode IN ('F', 'B')
      AND subject.subject_code = '00'
      AND (site.latitude IS NOT NULL AND site.longitude IS NOT NULL)
      AND (ACOS(least(1,COS(0.900395603976742)*COS(-0.02489630147495067)*COS(RADIANS(site.latitude))*COS(RADIANS(site.longitude))+COS(0.900395603976742)*SIN(-0.02489630147495067)*COS(RADIANS(site.latitude))*SIN(RADIANS(site.longitude))+SIN(0.900395603976742)*SIN(RADIANS(site.latitude))))*3963.1899999999996) <= '50')
    ORDER BY distance ASC LIMIT 10 OFFSET 0;

with query plan:
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                QUERY PLAN                                                                                │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Limit  (cost=17631.21..17631.24 rows=10 width=1160) (actual time=208.759..208.771 rows=10 loops=1)                                                                                │
│   Buffers: shared hit=126165 read=1818                                                                                │
│   ->  Sort  (cost=17631.21..17631.25 rows=14 width=1160) (actual time=208.758..208.769 rows=10 loops=1)                                                                                │
│         Sort Key: distances.distance                                                                                │
│         Sort Method: top-N heapsort  Memory: 28kB                                                                                │
│         Buffers: shared hit=126165 read=1818                                                                                │
│         ->  Hash Semi Join  (cost=11804.45..17630.94 rows=14 width=1160) (actual time=146.574..208.611 rows=148 loops=1)                                                                                │
│               Hash Cond: (course.id = course_site.course_id)                                                                                │
│               Buffers: shared hit=126165 read=1818                                                                                │
│               ->  Hash Join  (cost=9670.39..15446.28 rows=19191 width=1157) (actual time=63.974..125.011 rows=11327 loops=1)                                                                                │
│                     Hash Cond: (course.provider_id = provider.id)                                                                                │
│                     Buffers: shared hit=6953 read=1818                                                                                │
│                     ->  Hash Join  (cost=6912.32..12637.81 rows=19191 width=1156) (actual time=58.980..115.908 rows=11327 loops=1)                                                                                │
│                           Hash Cond: (course.id = distances.course_id)                                                                                │
│                           Buffers: shared hit=4370 read=1818                                                                                │
│                           ->  Seq Scan on course  (cost=0.00..5528.69 rows=74969 width=1144) (actual time=0.011..42.860 rows=74969 loops=1)                                                                                │
│                                 Buffers: shared hit=2961 read=1818                                                                                │
│                           ->  Hash  (cost=6672.43..6672.43 rows=19191 width=12) (actual time=58.547..58.550 rows=11327 loops=1)                                                                                │
│                                 Buckets: 32768  Batches: 1  Memory Usage: 743kB                                                                                │
│                                             Group Key: course_site_1.course_id                                                                                │
│                                             Batches: 1  Memory Usage: 1809kB                                                                                │
│                                             Buffers: shared hit=1409                                                                                │
│                                             ->  Hash Join  (cost=2505.35..4998.53 rows=23456 width=20) (actual time=19.971..39.421 rows=22869 loops=1)                                                                                │
│                                                   Hash Cond: (course_site_1.site_id = site_1.id)                                                                                │
│                                                   Buffers: shared hit=1409                                                                                │
│                                                   ->  Bitmap Heap Scan on course_site course_site_1  (cost=791.27..3222.52 rows=23590 width=8) (actual time=1.740..15.316 rows=23104 loops=1)                                                                                │
│                                                         Recheck Cond: (status = 'R'::text)                                                                                │
│                                                         Filter: ((publish = 'Y'::text) AND ((vac_status = 'F'::text) OR (vac_status = 'P'::text) OR (vac_status = 'B'::text)))                                                                                │
│                                                         Rows Removed by Filter: 48085                                                                                │
│                                                         Heap Blocks: exact=626                                                                                │
│                                                         Buffers: shared hit=688                                                                                │
│                                                         ->  Bitmap Index Scan on index_course_site_on_status  (cost=0.00..785.38 rows=71611 width=0) (actual time=1.677..1.677 rows=71189 loops=1)                                                                                │
│                                                               Index Cond: (status = 'R'::text)                                                                                │
│                                                               Buffers: shared hit=62                                                                                │
│                                                   ->  Hash  (cost=1264.07..1264.07 rows=36000 width=20) (actual time=17.969..17.970 rows=35914 loops=1)                                                                                │
│                                                         Buckets: 65536  Batches: 1  Memory Usage: 2336kB                                                                                │
│                                                         Buffers: shared hit=721                                                                                │
│                                                         ->  Seq Scan on site site_1  (cost=0.00..1264.07 rows=36000 width=20) (actual time=0.011..8.971 rows=35914 loops=1)                                                                                │
│                                                               Filter: ((latitude IS NOT NULL) AND (longitude IS NOT NULL) AND ((address1 <> ''::text) OR (postcode <> ''::text)))                                                                                │
│                                                               Rows Removed by Filter: 291                                                                                │
│                                                               Buffers: shared hit=721                                                                                │
│                     ->  Hash  (cost=2660.81..2660.81 rows=7781 width=5) (actual time=4.939..4.939 rows=7781 loops=1)                                                                                │
│                           Buckets: 8192  Batches: 1  Memory Usage: 368kB                                                                                │
│                           Buffers: shared hit=2583                                                                                │
│                           ->  Seq Scan on provider  (cost=0.00..2660.81 rows=7781 width=5) (actual time=0.010..3.260 rows=7781 loops=1)                                                                                │
│                                 Buffers: shared hit=2583                                                                                │
│               ->  Hash  (cost=2133.40..2133.40 rows=53 width=16) (actual time=82.086..82.091 rows=1356 loops=1)                                                                                │
│                     Buckets: 2048 (originally 1024)  Batches: 1 (originally 1)  Memory Usage: 80kB                                                                                │
│                     Buffers: shared hit=119212                                                                                │
│                     ->  Nested Loop  (cost=28.11..2133.40 rows=53 width=16) (actual time=14.790..81.569 rows=1356 loops=1)                                                                                │
│                           Buffers: shared hit=119212                                                                                │
│                           ->  Nested Loop  (cost=27.82..2071.32 rows=161 width=20) (actual time=14.589..49.099 rows=18388 loops=1)                                                                                │
│                                 Join Filter: (course_site.course_id = site_statuses_course_join.course_id)                                                                                │
│                                 Buffers: shared hit=64048                                                                                │
│                                 ->  Nested Loop  (cost=27.53..2030.47 rows=94 width=12) (actual time=14.583..38.029 rows=2296 loops=1)                                                                                │
│                                       Buffers: shared hit=54393                                                                                │
│                                       ->  Nested Loop  (cost=27.25..1841.37 rows=375 width=16) (actual time=0.353..33.323 rows=2547 loops=1)                                                                                │
│                                             Join Filter: (course_site.course_id = course_1.id)                                                                                │
│                                             Buffers: shared hit=46752                                                                                │
│                                             ->  Nested Loop  (cost=26.83..1435.90 rows=573 width=8) (actual time=0.346..25.608 rows=3383 loops=1)                                                                                │
│                                                   Buffers: shared hit=33220                                                                                │
│                                                   ->  Nested Loop  (cost=26.54..632.45 rows=1838 width=4) (actual time=0.333..4.212 rows=10655 loops=1)                                                                                │
│                                                         Buffers: shared hit=566                                                                                │
│                                                         ->  Seq Scan on subject  (cost=0.00..1.56 rows=1 width=8) (actual time=0.009..0.014 rows=1 loops=1)                                                                                │
│                                                               Filter: (subject_code = '00'::text)                                                                                │
│                                                               Rows Removed by Filter: 44                                                                                │
│                                                               Buffers: shared hit=1                                                                                │
│                                                         ->  Bitmap Heap Scan on course_subject  (cost=26.54..612.51 rows=1838 width=8) (actual time=0.321..2.544 rows=10655 loops=1)                                                                                │
│                                                               Recheck Cond: (subject_id = subject.id)                                                                                │
│                                                               Heap Blocks: exact=555                                                                                │
│                                                               Buffers: shared hit=565                                                                                │
│                                                               ->  Bitmap Index Scan on index_course_subject_on_subject_id  (cost=0.00..26.08 rows=1838 width=0) (actual time=0.270..0.271 rows=10655 loops=1)                                                                                │
│                                                                     Index Cond: (subject_id = subject.id)                                                                                │
│                                                                     Buffers: shared hit=10                                                                                │
│                                                   ->  Index Scan using "IX_course_site_course_id" on course_site  (cost=0.29..0.43 rows=1 width=4) (actual time=0.002..0.002 rows=0 loops=10655)                                                                                │
│                                                         Index Cond: (course_id = course_subject.course_id)                                                                                │
│                                                         Filter: ((status = ANY ('{N,R}'::text[])) AND (vac_status <> ''::text) AND (status = 'R'::text) AND (publish = 'Y'::text))                                                                                │
│                                                         Rows Removed by Filter: 1                                                                                │
│                                                         Buffers: shared hit=32654                                                                                │
│                                             ->  Index Scan using "PK_course" on course course_1  (cost=0.42..0.70 rows=1 width=8) (actual time=0.002..0.002 rows=1 loops=3383)                                                                                │
│                                                   Index Cond: (id = course_subject.course_id)                                                                                │
│                                                   Filter: ((discarded_at IS NULL) AND (study_mode = ANY ('{F,B}'::text[])) AND (qualification = ANY ('{1,2,3,4}'::integer[])))                                                                                │
│                                                   Rows Removed by Filter: 0                                                                                │
│                                                   Buffers: shared hit=13532                                                                                │
│                                       ->  Index Scan using "PK_ucas_provider" on provider provider_1  (cost=0.28..0.50 rows=1 width=4) (actual time=0.001..0.001 rows=1 loops=2547)                                                                                │
│                                             Index Cond: (id = course_1.provider_id)                                                                                │
│                                             Filter: ((discarded_at IS NULL) AND (recruitment_cycle_id = 3))                                                                                │
│                                             Rows Removed by Filter: 0                                                                                │
│                                             Buffers: shared hit=7641                                                                                │
│                                 ->  Index Scan using "IX_course_site_course_id" on course_site site_statuses_course_join  (cost=0.29..0.40 rows=3 width=8) (actual time=0.001..0.003 rows=8 loops=2296)                                                                                │
│                                       Index Cond: (course_id = course_subject.course_id)                                                                                │
│                                       Buffers: shared hit=9655                                                                                │
│                           ->  Index Scan using "PK_ucas_campus" on site  (cost=0.29..0.39 rows=1 width=4) (actual time=0.002..0.002 rows=0 loops=18388)                                                                                │
│                                 Index Cond: (id = site_statuses_course_join.site_id)                                                                                │
│                                 Filter: ((latitude IS NOT NULL) AND (longitude IS NOT NULL) AND ((acos(LEAST('1'::double precision, (((('0.6211074934385727'::double precision * cos(radians(latitude))) * cos(radians(longitude))) + (('-0.015466475043944938'::double precision * cos(radians(latitude))) * sin(radians(longitude)))) + ('0.7835727597001878'::double precision * sin(radians(latitude)))))) * '3963.1899999999996'::double precision) <= '50'::double precision)) │
│                                 Rows Removed by Filter: 1                                                                                │
│                                 Buffers: shared hit=55164                                                                                │
│ Planning:                                                                                │
│   Buffers: shared hit=232                                                                                │
│ Planning Time: 29.407 ms                                                                                │
│ Execution Time: 210.312 ms                                                                                │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
(105 rows)

Time: 241.081 ms

It's worth noting here that we join from course to course_site twice which would appear to be necessary given that both are INNER JOINs.
Changes proposed in this pull request

Attempt to avoid the double join to site_status. These are the most expensive parts of the query. I've changed the within and by_distance scopes to join to site via site_status so that they can use the existing join.
Attempt to remove the sub-query that was introduced to avoid duplicates when filtering by provider name.

Potential changes in future
There are some other things that we should look at that could give short-term gains:

Investigate whether the autosuggest for locations can be tuned.
Look at 1+n queries. Looking at the SQL logs there do seem to be a couple of 1+n queries though they don't appear to be adding a significant performance overhead (perhaps because we are paginating just 10 results at a time).
It looks like there is significant delay when there are no matches for a given location/filter set. Needs investigation.

In the longer term there may be bigger improvements that we can make as part of a more general rearchitecting of TTAPI, Find and other services. For example:

Find could maintain its own Postgresql database, synched from TTAPI. This database could be highly customised to its specific needs so it might represent a highly denormalised version of the course data.
We could potentially use a different kind of data store for course data in Find like Elasticsearch or similar. Again the data would be structured specifically to service course searches.

Guidance to review
Removing the extra join to the course_site table gives different results to the original query. It took me a while to work out why this happens. However I think the original query was returning some courses that it should not have (but I may be misunderstanding the requirement).
The relevant parts of the original query are (some details omitted):
SELECT
  course.*
FROM
  "course"
  INNER JOIN "course_site" ON "course_site"."course_id" = "course"."id"
  INNER JOIN "course_site" "site_statuses_course_join" ON "site_statuses_course_join"."course_id" = "course"."id"
  INNER JOIN "site" ON "site"."id" = "site_statuses_course_join"."site_id"
WHERE "course_site"."status" IN ('N', 'R')
  AND some-trigonometry-function-that-works-out-distance("site.lat", "site.lng") < 50;

Here you can see there are two joins to course_site the second of which also joins to site (so that the closest location can be retrieved - not shown). The first join has a condition on course_site.status that is meant to filter out inactive sites (I believe). The second join has a condition on site lat/lng (within 50 miles of the search location). It's possible a course to have one site that matches the first condition and another that matches the second. If we combine these two joins and their associated conditions then that course no longer matches because neither of its sites satisfies both conditions. So I think that combining the joins has accidentally fixed a bug in the search logic. Does this make sense?
In terms of performance I'm seeing measurable benefits performance when doing a location based search. Not when doing an across England search. e.g. the location query above has a top speed (on my machine) of about 210ms. The optimised one is about 125ms. Because the same query is again run to count the total the gains are doubled.
My main concern is whether the results are correct. What is the best way to test this?
My changes broke pretty much all of the tests for CourseSearchService (they rely on stubs/mocks so are fairly tied to the implementation). I've tried to fix them 'as there are' rather than make any radical changes here.
Checklist

[x] Make sure all information from the Trello card is in here
[x] Attach to Trello card
[x] Rebased master
[x] Cleaned commit history
[x] Tested by running locally

Find could maintain its own Postgresql database, synched from TTAPI. This database could be highly customised to its specific needs so it might represent a highly denormalised version of the course data.
We could potentially use a different kind of data store for course data in Find like Elasticsearch or similar. Again the data would be structured specifically to service course searches.

💯

Really good investigation and write-up @stevehook 👍🏼
Reasoning makes sense to me. Would definitely be worth @gpeng and anyone else with more Find experience taking a look over this as well.

Closing this in favour of https://github.com/DFE-Digital/teacher-training-api/pull/2205 and https://github.com/DFE-Digital/teacher-training-api/pull/2197


change the key name of required.attributes

change the key name of required.attributes to  gobblin.template.required_attributes

Changes Unknown when pulling f12c0755606194764703c5d2275d5912f2e95b65 on autumnust:template-dev into ** on linkedin:master**.

LGTM, but this will conflict with a fix I had to make regarding the name of ResourcesBasedTemplate. Please fix that and let me know before committing.

@ibuenros  fixed.

Coverage decreased (-0.006%) to 46.549% when pulling b52217954b43ceb8f57987ff6d60507a5a21260f on autumnust:template-dev into 1dc980eec5969f4dfd75b808e5913c135c10538f on linkedin:master.


Set up AWS

Set up an account and validate that these elements work as expected:

[ ] Clojure server
[ ] Images
[ ] Videos

So, images are served up just fine with s3 and there's AWS-transcode for creating pipelines to standardise video-format etc. ATM, I'm using Elastic Beans with Tomcat to serve the site


"post_likers" function returns incorrect amount of likers on a post

Hi there,
The post_likers function does not return the exact amount of likers on any post with more than 128 likes.
post = "https://www.instagram.com/p/CDbt2ikF2q9/"

like_count = bot.like_count(post)
print(like_count)
response = bot.post_likers(post, limit=like_count)
print(len(response))

returns:
841
504

Really enjoy the bot you've written so far - nice work!

First of all thanks @grantjayy for opening this issue. I've checked what you said with different posts, and I found that the specific post you mentioned have a big difference between the number of likers and there usernames. But when I tried other posts the difference is not that big and it's negligible. The screenshot of the output/results the image is attached. And again thanks for using my api and opening this issue.

Thankful for this app
certainly in a post of 9000 like it only brings me 3000

Thankful for this app
certainly in a post of 9000 like it only brings me 3000


freedv: <IP_ADDRESS> -> 1.8.4

Description of changes
Supersedes #198804.
https://github.com/drowe67/freedv-gui/releases/tag/v1.8.4
https://github.com/drowe67/LPCNet/releases/tag/v0.3
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This looks good to me. I would have split up the commits, but that may be just a matter of taste.
I suggest that @markuskowa should have a quick glance at the changes to the codec2 derivation. Now that the circular dependency has been resolved, it may or may not make sense to just enable LPCNet support (-LPCNET:BOOL=ON) unconditionally.

Result of nixpkgs-review pr 198852 run on x86_64-linux 1

2 packages built:

freedv
lpcnetfreedv

Successfully tested basic functionality of the freedv binary on NixOS (x86-64).

@wegank, as you appear to be using FreeDV on Darwin, I have no objection to sharing maintenance responsibility. Feel free to add yourself to the list of maintainers at the next opportunity.


Using .htaccess with .NET Core

I'm setting up a .NET Core 2.2 app on my Linux/Apache2 server but can't get .htaccess to work. I placed .htaccess at the root of the application, alongside all the other DLLs, and it contains URL redirect rules. However, those rules aren't working.
I saw this article about URL Rewrite Middleware
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/url-rewriting?view=aspnetcore-2.2
It states:

Use URL Rewriting Middleware when you're unable to use the following approaches: Apache mod_rewrite module on Apache Server

So my question is, can I use .htaccess and mod_rewrite on Linux with .NET Core or not?

Typically you'd use Kestrel as the web server on Linux, not Apache. I believe you can use Apache as a reverse proxy, or to host additional static assets, but not for hosting the .NET Core app.
The ASP.NET Core Rewrite Middleware can load certain compatible rewrite rules, such as ones from IIS or Apache. So, instead of using Apache mod_rewrite, you can load the same type of rules into the ASP.NET Core feature. (There could conceivable be some differences, because it's unlikely to be 100% compatible.)

Here's some more info on using Apache with ASP.NET Core: https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-apache?view=aspnetcore-2.2
And some of the other docs above/below that in the Table of Contents should be illuminating.

They highly recommend running a web server like Apache2 behind Kestrel; otherwise Kestrel on its own can only host a single application on the server.
In the doc, they set the rewrite rules directly in the Apache2 config file. You say you can load rewrite rules from the middleware, but they say Middleware's performance is lower than handling rewrites at the Apache2 or IIS level and should only be done if it can't be done on the web server directly.
Apart from these two options, can .htaccess be used or not?

They highly recommend running a web server like Apache2 behind Kestrel

Correction: you run Kestrel behind Apache2 to enable hosting multiple applications.

Here's my understanding:
If you want to use an actual .htaccess file, you need to use Apache at whatever node/server you want to use it.
If you want to just use the contents of a .htaccess file (say, because you already have one from a service you're trying to migrate), you can use ASP.NET Core's built-in support for loading that file with the RewriteMiddleware.

If you want to use an actual .htaccess file, you need to use Apache at whatever node/server you want to use it.
So how do I do that?

If I just want the content in a middleware, I'd just reconfigure the rewrites in middleware format. Looking at other options before going that route.

@mysteryx93 can you share some info on what you're trying to set up? What server(s) are in play, for what purpose, or any other info?

.NET Core on Apache2 on Ubuntu.
ProxyPass / http://<IP_ADDRESS>:5000/
ProxyPassReverse / http://<IP_ADDRESS>:5000/

Now that I think of it, Apache2 should process .htaccess BEFORE forwarding anything to .NET Core, so there's no way .NET Core can conflict with it if it's setup right. After doing some research, I also realized .htaccess lookup is disabled by default for performance and security reasons. They also recommended to do the rewrites directly in the Apache2 config file to avoid a performance hit. Doing so in that way is working.
That doesn't explain why the redirect middleware isn't working though; which I posted as a separate issue.

That doesn't explain why the redirect middleware isn't working though; which I posted as a separate issue.

This is https://github.com/aspnet/AspNetCore/issues/10543 right?

Apache2 should process .htaccess BEFORE forwarding anything to .NET Core, so there's no way .NET Core can conflict with it if it's setup right.

Correct. We don't control how Apache2 processes .htaccess.


/k argument ignored from 'commandline' setting

Environment
Windows build number: 10.0.18362.207
Windows Terminal version (if applicable): 0.2.21715.0

Steps to reproduce
I have setenv.bat file in S:\dir.
Add new command in profiles.json:
with     "commandline": "cmd.exe **/k S:\\dir\\setenv.bat**", like this:
{
      "acrylicOpacity": 0.75,
      "closeOnExit": true,
      "colorScheme": "Campbell",
      "commandline": "cmd.exe **/k S:\\dir\\setenv.bat**",
      "cursorColor": "#FFFFFF",
      "cursorShape": "bar",
      "fontFace": "Consolas",
      "fontSize": 10,
      "guid": "{0caa0dad-35be-5f56-a8ff-afceeeaa6101}",
      "historySize": 9001,
      "icon": "ms-appx:///ProfileIcons/{0caa0dad-35be-5f56-a8ff-afceeeaa6101}.png",
      "name": "cmd custom",
      "padding": "0, 0, 0, 0",
      "snapOnInput": true,
      "startingDirectory": "S:\\omaha\\src",
      "useAcrylic": true
    },

Expected behavior
I expect "cmd custom" to launch cmd.exe /k s:\dir\setenv.bat.
Actual behavior
setenv.bat is never launched.

I think my issue was not replacing the guid field.

Cool. Resolved, then?


fix(package.json):  repository key

we need to update that, IDK but I'm convinced that npm publish without this update


With HttpCache enabled, adding an Interceptor causes ApolloNetworkException

Summary
It seems this is related to #1786. On a fresh install with OkHttpClient:
OkHttpClient.Builder().apply {
     addInterceptor(HttpLoggingInterceptor().setLevel(HttpLoggingInterceptor.Level.BODY))
}

If a request does not have a cached response yet, this error is thrown
com.apollographql.apollo.exception.ApolloNetworkException: Failed to execute http call
        at com.apollographql.apollo.internal.interceptor.ApolloServerInterceptor$executeHttpCall$1.onFailure(ApolloServerInterceptor.kt:103)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:532)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: java.io.IOException: canceled due to java.lang.IllegalStateException: closed
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:530)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)     	Suppressed: java.lang.IllegalStateException: closed
        at okio.RealBufferedSource.read(RealBufferedSource.kt:185)
        at com.apollographql.apollo.cache.http.ResponseBodyProxy$ProxySource.read(ResponseBodyProxy.java:75)

Version
2.5.3
Description
Full stack trace:
com.apollographql.apollo.exception.ApolloNetworkException: Failed to execute http call
        at com.apollographql.apollo.internal.interceptor.ApolloServerInterceptor$executeHttpCall$1.onFailure(ApolloServerInterceptor.kt:103)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:532)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: java.io.IOException: canceled due to java.lang.IllegalStateException: closed
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:530)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)     	Suppressed: java.lang.IllegalStateException: closed
        at okio.RealBufferedSource.read(RealBufferedSource.kt:185)
        at com.apollographql.apollo.cache.http.ResponseBodyProxy$ProxySource.read(ResponseBodyProxy.java:75)
        at okio.RealBufferedSource.request(RealBufferedSource.kt:206)
        at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:247)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)

Note that I am also seeing a related(?) issue with the following interceptor:
class OAuthInterceptor(
    private val restRequestHelper: RestRequestHelper,
    private val sessionRepository: SessionRepository
) : Interceptor {

    override fun intercept(chain: Interceptor.Chain): Response {
        val newRequest = chain.request().newBuilder()
            .addHeader("Authorization", restRequestHelper.getAuthorizationHeader(sessionRepository.accessToken))
            .addHeader("User-Agent", restRequestHelper.getHeaderUserAgentString())
            .build()

        return chain.proceed(newRequest)
    }
}

    com.apollographql.apollo.exception.ApolloNetworkException: Failed to execute http call
        at com.apollographql.apollo.internal.interceptor.ApolloServerInterceptor$executeHttpCall$1.onFailure(ApolloServerInterceptor.kt:103)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:525)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: okhttp3.internal.http2.StreamResetException: stream was reset: INTERNAL_ERROR
        at okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.kt:148)
        at okhttp3.internal.http2.Http2ExchangeCodec.readResponseHeaders(Http2ExchangeCodec.kt:96)
        at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:106)
        at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:79)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at com.apollographql.apollo.cache.http.HttpCacheInterceptor.networkFirst(HttpCacheInterceptor.java:89)
        at com.apollographql.apollo.cache.http.HttpCacheInterceptor.intercept(HttpCacheInterceptor.java:51)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:221)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at com.doximity.doximitydroid.sources.auth.OAuthInterceptor.intercept(OAuthInterceptor.kt:18)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at com.apollographql.apollo.ApolloClientAwarenessInterceptor.intercept(ApolloClientAwarenessInterceptor.kt:18)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
        at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:764)

Hi 👋
Is there any chance you can share a reproducer project somewhere ? I tried adding a HTTP cache and interceptor to the tutorial: https://github.com/apollographql/apollo-android-tutorial/tree/reproducer-2912
But haven't been able to reproduce so far.

@martinbonnin I figured out the problem - we had a duplicate HttpLoggingInterceptor set on our OkHttpClient. If I remove the duplicate, http caching works fine. We have a number of different OkHttpClients that build off of a "base" OkHttpClient. In one case, both the base client and a "sub" client were adding a HttpLoggingInterceptor, resulting in a total of 2
So I'm not sure if this is an Apollo issue or not? Seems like it should be able to handle this case potentially
I looked at your reproducer-2912 branch on the tutorial repo. I don't have access to push up, but if you add a second HttpLoggingInterceptorhere, you'll see a IllegalStateException: closed like I posted in the issue description
In any case, thank you for the quick response!

Thanks for the follow up! If that's ok with you I'll close this issue as having 2 HttpLoggingInterceptor sounds like a edge case? We can always reopen if that happens in other cases.

Sounds good to me, thanks @martinbonnin


Add a dotDescription property to TKStateMachine

This can greatly improve understanding what a state machine looks like.
Here is the visualization of the relationship state machine of the spec:

This is clever. Merged

:+1:


fix: transfers now serialize consistently

Signed-off-by: Sean Tedrow<EMAIL_ADDRESS>Description:
Transfers in now serialize in a consistent order which is independent of the order in which they were added to the TransferTransaction.
All *Id types are now comparable.
Related issue(s):
Fixes #769
Checklist

[X] Documented (Code comments, README, etc.)
[X] Tested (unit, integration, etc.)

Also, SonarCloud has some legitimate points.


TextField text cursor going to the first position after value update

This bug is android only, I have a Slider and a TextField that have the same ngModel attribute reference and when the value of that attribute is updated, the cursor goes to the first position of the input, no matters if the user use the textfield or the slider.
Sample project:
https://play.nativescript.org/?template=play-ng&id=O0oLGQ&v=1

Hi @yassern,
Thank you for contacting us.
I tested this case and found that it is reproducible also in a non-Angular project. For further info, please keep track on the issue.

This issue was moved to NativeScript/NativeScript#5873


Post Weights Checklist

Here are some things that we should do once initial weights are "completed"

[ ] Write down all changed APIs to pallets (additional parameters, additional traits, etc)

[ ] Write down all changed functionality to pallets (new functions? functions do different things?)

[ ] Write down all assumptions about maximums taken in weight calculations.

[ ] Double check all weights one last time (sanity check)

[ ] Ensure Polkadot JS API is updated for changes APIs and Functionality

[ ] Check operational dispatches if they should be operational, and check if any new dispatches can be operational.

[ ] Make sure we can save a blockchain with full blocks using only operational transactions.

[ ] Fee calculation from block data: https://github.com/paritytech/substrate/issues/6022

[ ] move all weights to saturating_xxx.

[ ] collect open questions regarding future weight work (e.g. "how to do transitive weights well?", "how to help others building with substrate to do weights well?")

move all weights to saturating_xxx.

an option could also be to use a type which saturate by itself, std provides, https://doc.rust-lang.org/core/num/struct.Wrapping.html
Somebody implemented https://crates.io/crates/saturating
Though I think manual saturation is fine.


New Chat - There is NO focus when opening tab via FAB button

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to URL https://staging.new.expensify.com/
Login with any account
Click on FAB button -> New Chat

Expected Result:
Cursor presents in the search field
Actual Result:
The cursor is missing in the search field.
New Group - the same behavior.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[x] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] Windows / Chrome
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://platform.applause.com/services/links/v1/external/5bae2830f9fdeb089985848c3383dbcd2e3111818892065011d40b0c64502352
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

Proposal
Please re-state the problem that we are trying to solve in this issue.
New Chat - There is NO focus when opening tab via FAB button
What is the root cause of that problem?
We're using OptionSelector to show the TextInput and Options:
https://github.com/Expensify/App/blob/9e5b119f0005088a3870ab4eb53e6b4795d9b8d4/src/pages/NewChatPage.js#L240-L243
But since we implemented skeleton, the input will need a little bit times to focus.
What changes do you think we should make in order to solve the problem?

We should add shouldDelayFocus to our OptionsSelector:
  <OptionsSelector
                                // autoFocus
                                shouldDelayFocus={shouldDelayFocus}

What alternative solutions did you explore? (Optional)
NA

I will prioritize this today.

I don't think this is a bug, this is expected behavior, and the cursor will appear when the user clicks on the name, email, or phone number field under new chat and new group.


Update Grains exercise

https://forum.exercism.org/t/odd-arithmetic-in-grains/8662/23
The example for Grains needs to be updated so the Square and Total functions return a string.
The exercise difficulty needs to be updated to medium.
The "grains on square 64" and "returns the total number of grains on the board" tests need to be updated to use the correct expected values as strings.

Apparently I closed it as a result of misinterpretation involving 269 being merged.

Addressed via #326
I believe this is now "hard" as it uses strings to get beyond the integer limitations, and could theoretically be solved using 8 bit vim, if there were such a thing.

If you’re referring to the difficulty displayed on the site, a difficulty of 5 is a medium exercise. It needs to be 7 or higher to be a hard exercise.


Dependencies warning message is missing module name

Warning message printed for my imported 6.2 theme:
Warning: You must install the correct dependencies, please run npm i --save-dev from your theme directory.
It does not tell me what dependency I'm missing though. I was able to figure out that my theme uses ruby-sass, so I was missing the gulp-ruby-sass dependency.
I see they are defined here:
https://github.com/liferay/liferay-themes-sdk/blob/master/packages/liferay-theme-tasks/lib/doctor.js#L121-L139
Looks like the modulename isn't being passed:
https://github.com/liferay/liferay-themes-sdk/blob/master/packages/liferay-theme-tasks/lib/doctor.js#L164
I think perhaps this line: https://github.com/liferay/liferay-themes-sdk/blob/50790611b44f946bfdbe421607ea762fdf24bd26/packages/liferay-theme-tasks/lib/doctor.js#L164 should be:
colors.cyan('npm i --save-dev ' + moduleName),
Tested locally. Confirmed that is the fix.

Sent fix https://github.com/liferay/liferay-themes-sdk/pull/79


ModuleNotFoundError: No module named 'aiohttp'

(Filling out the following details about bugs will help us solve your issue sooner.)
Reproducible in:
pip freeze | grep slack
python --version
sw_vers && uname -v # or `ver`

The Slack SDK version
slack-sdk==3.18.1
Python runtime version
Python == 3.7
OS info
ProductName:    macOS
ProductVersion: 12.4
BuildVersion:   21F79
Steps to reproduce:
(Share the commands to run, source code, and project settings (e.g., setup.py))
1.Import both WebClient and SlackApiError from slack package  to my Apache Airflow DAG
2.Apache Airflow sends an error
3.
Expected result:
Apache Airflow DAG succesfully loaded to webserver UI
Actual result:
Broken DAG: [/opt/airflow/dags/trigger_dag.py] Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/slack/__init__.py", line 7, in <module>
    from slack_sdk.rtm import RTMClient  # noqa
  File "/home/airflow/.local/lib/python3.7/site-packages/slack_sdk/rtm/__init__.py", line 16, in <module>
    import aiohttp
ModuleNotFoundError: No module named 'aiohttp'

Requirements
For general questions/issues about Slack API platform or its server-side, could you submit questions at https://my.slack.com/help/requests/new instead. :bow:
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

Hi @PapiG0nz0 - aiohttp is an external module used by python-slack-sdk. I'm not so familiar with the Apache Airflow DAQ environment, but you will need to make sure that module is available for the Python SDK to work.

@PapiG0nz0
I don't think that your Airflow DAG needs the RTM client. If you are developing new code, you can use slack_sdk modules instead.
This SDK provides slack modules for smooth migration from slackclient v2 package. Refer to the v3 migration guide for details. If you have existing code, you can go with either of:

Migrate to slack_sdk modules instead of importing slack
Add aiohttp to your dependencies if you need to import slack modules for some reason

I hope this helps.


Added ODR server side notification.

Support ODR onBlobReplicaReplicated notification.

Codecov Report
Patch coverage has no change and project coverage change: -59.21 :warning:

Comparison is base (fd9943f) 73.27% compared to head (1853b43) 14.07%.

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #2482       +/-   ##
=============================================
- Coverage     73.27%   14.07%   -59.21%
+ Complexity    11089     2360     -8729
=============================================
  Files           783      783
  Lines         62217    62226        +9
  Branches       7624     7626        +2
=============================================
- Hits          45591     8757    -36834
- Misses        14126    52962    +38836
+ Partials       2500      507     -1993

Impacted Files
Coverage Δ

.../github/ambry/notification/NotificationSystem.java
0.00% <ø> (ø)

...ithub/ambry/commons/LoggingNotificationSystem.java
0.00% <0.00%> (-53.13%)
:arrow_down:

...a/com/github/ambry/network/LocalNetworkClient.java
0.00% <0.00%> (-96.78%)
:arrow_down:

.../java/com/github/ambry/protocol/AmbryRequests.java
0.00% <0.00%> (-76.19%)
:arrow_down:

... and 624 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Error when running testing 'FileNotFoundError: [Errno 2] No such file or directory: '/root/.pyrouge/settings.ini''

I get this error when i am running testing

Traceback (most recent call last):
File "train.py", line 353, in
test(args, device_id, cp, step)
File "train.py", line 216, in test
trainer.test(test_iter,step)
File "/content/BertSum/src/models/trainer.py", line 297, in test
rouges = test_rouge(self.args.temp_dir, can_path, gold_path)
File "/content/BertSum/src/others/utils.py", line 79, in test_rouge
r = pyrouge.Rouge155(temp_dir=temp_dir)
File "/content/BertSum/src/others/pyrouge.py", line 123, in init
self.__set_rouge_dir(rouge_dir)
File "/content/BertSum/src/others/pyrouge.py", line 439, in __set_rouge_dir
self._home_dir = self.__get_rouge_home_dir_from_settings()
File "/content/BertSum/src/others/pyrouge.py", line 453, in __get_rouge_home_dir_from_settings
with open(self._settings_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/root/.pyrouge/settings.ini'

Here is a direct public link to the notebook I am working on
https://colab.research.google.com/drive/1vKYu-RPR5FJ1ZukTC7JDcfDYK6bm45eq

I see it's been addressed here
https://github.com/nlpyang/BertSum/issues/35

I have a Question. How do you solve that Problem? I'm using google Cloab and ich don't know how i fix that issue.


Release 2.0.0-beta.2 using broccoli-uglify-sourcemap 2.0.0-beta.2

Would you please release a 2.0.0-beta.2 version of this package, which uses broccoli-uglify-sourcemap 2.0.0-beta.2? I believe broccoli-uglify-sourcemap 2.0.0-beta.2 fixes a problem we're having, so I'd love to try it out and run some builds. Will provide feedback!

@rwjblue @Turbo87

Scratch that, yarn will resolve up to beta.2. Neat.


should deactivate only when updating volume

https://github.com/rancher/rancher/issues/8357

LGTM


Support send mail using text/html

The mail integration have not supported text/html, only text/plain now.
from: https://github.com/argoproj-labs/argocd-notifications/discussions/291

This is great! thanks @ryota-sakamoto


Label api improvements

Add support for the new label API improvements as described in https://developer.github.com/changes/2018-02-22-label-description-search-preview/ and addresses #856
This PR adds:

support for the Description and Default fields of the labels
search label

N.B. search label result text highlighting is no implemented yet. I can add it to this PR or open another one but since this is my first contribution to this project (and my first public go PR),  I would like go get feedback early.

Should be all good now!

@gjacquet - it looks like your commits are coming from two different email addresses, and the cla/google check might not be happy until both email addresses have signed CLAs or are from the same GitHub account.
So you may be able to associate both email addresses with your GitHub account by visiting https://github.com/settings/emails and clicking Add email address.
If that doesn't fix the problem, then you probably will have to follow the CLA link (see comments from @googlebot above) in an incognito window from both email addresses and sign the CLA. If that doesn't work, then we may need to pull in the expert.

You are absolutely right. The doc does not explicitly states it is required like q for instance so I assumed it was not required.
I did get a 422 when querying without repo id. I’ll fix that in a minute.

Thanks for checking.
Did you see my earlier https://github.com/google/go-github/pull/866#discussion_r178421939? Do you think it's a good idea?

@shurcooL now that we’ve established that repoID is mandatory, your suggestion makes a lot of sense.

@shurcooL I have just updated the comment based on your suggestion.

OK, thanks, @gjacquet... @willnorris can clear the @googlebot issue for us... possibly just by squashing and merging for us... I'll leave it up to him.


Align cassandra version witch create schema cassandra version

Signed-off-by: Pavol Loffay<EMAIL_ADDRESS>

This change is


Add cpms e2e test

Which issue this PR addresses:
Follow-up for ARO-8836, #3660
What this PR does / why we need it:
Adds E2E test for ARO Operator CPMS handling:

Ensures it is Inactive
Ensures it is reset to Inactive if activated

To achieve this, the openshift/client-go vendored dependency was updated to the release-4.12 branch.
Test plan for issue:

[x] E2E passes, with the test behaving as expected

Is there any documentation that needs to be updated for this PR?
No
How do you know this will function as expected in production?


Zombie workflows (aka "stuck workflows")

A "zombie workflow" is one that starts but does not complete. Pods are scheduled and run to completion, but the workflow is not subsequently updated.
It is as if the workflow controller never sees the pod changes.
Impacted users:

@BlackRider97
@AbhishekMallick
@jayz28
@ebr
@jkgeyti
@TekTimmy

All users have been running very large workflows.
Typically:

Zombie workflows are running 5000+ pods at once.
insignificant pod change is not seen in the controller logs.
Deadline exceeded is seen in the logs. Increasing the CPU and memory on the Kubernetes master node may fix this.

Things that don't appear to work (rejected hypothesis):

Changing --burst or --qps, QPS settings.
Changing --workflow-workflow or --pod-workers settings. This only impacts concurrent processing.
Increasing the controller's memory or CPU.
Setting ALL_POD_CHANGES_SIGNIFICANT=true. Hypothesis: we're missing significant pod changes.
Setting INFORMER_WRITE_BACK=false.

Questions:

Every 20m workflow should re-reconcile. Did waiting fix it?
Did restarting the controller fix it?
Is that pods didn't start? Or that we don't see their completion?
Does the zombie workflow have the 'workflows.argoproj.io/completed: true` label?

Users should try the following:

Run argoproj/workflow-controller:v2.11.7 - this is faster than v2.11.6 and all previous versions. Suitable for production.
Set GC settings: https://argoproj.github.io/argo/cost-optimisation/#limit-the-total-number-of-workflows-and-pods
Delete any old completed workflows.
Run argoproj/workflow-controller:latest.
Run argoproj/workflow-controller:mot with env MAX_OPERATION_TIME=30s. Make sure it logs defaultRequeueTime=30s maxOperationTime=30s. Hypothesis: we need more time to schedule pods.
Run argoproj/workflow-controller:easyjson.  Hypothesis: JSON marshaling is very slow.

If none of this works then we need to investigate deeper.
Related:

#4239
#4048
#4398
#4423
#4560

Not sure if relevant: immediately following the controller restart, Kubelet operation rate for inspect_container and inspect_image increases to ~10x higher than the usual baseline. It subsides eventually. No other metrics are correlated, in kubelet or elsewhere:

(If this sounds like a red herring or orthogonal, I have no problem deleting this message or opening a separate issue)

Testing this on :latest.

a workflow is started where withItems is used on a step template, expecting to spawn 8172 nodes with children, for a total of ~40K nodes in the workflow
iteration begins and pods are started, but pauses at item 1256 with the same symptom as originally described (parent node shows "running", one child node has completed, but no further child nodes are being created)[1]
15 minutes later, some of the first iterations have completed, and item 1269 us being processed

further checks (about every 10 minutes) show that node count is going up slowly, and pauses for several minutes with the same symptom [1]
eventually the work stalls around item 1993, with no new pods being created for this workflow, even though the cluster has ample capacity.
Deadline exceeded messages are still being logged
[1]
  │ └───✔ get-work-unit
  ├─● run-pipeline(1988:1988)
  │ └───✔ get-work-unit
  ├─● run-pipeline(1989:1989)
  │ └───✔ get-work-unit
  ├─● run-pipeline(1990:1990)
  │ └───✔ get-work-unit
  ├─● run-pipeline(1991:1991)
  │ └───✔ get-work-unit
  ├─● run-pipeline(1992:1992)
  │ └───✔ get-work-unit
  └─● run-pipeline(1993:1993)
    └───✔ get-work-unit

Testing this on :latest.

Thank you. Can you try this?
Run argoproj/workflow-controller:mot.with env MAX_OPERATION_TIME=30s. Make sure it logs defaultRequeueTime=30s maxOperationTime=30s. Hypothesis: we need more time to schedule pods.

I did have INFORMER_WRITE_BACK=false both set and not set, as I'm also testing #4565 in parallel :)
Seeing same behaviour on :no-sig. I can try :mot now as you describe above.

Here's another perhaps relevant point: a workflow that is both: 1) stuck, AND 2) has a pod that's running, but is "frozen" (for a reason unrelated to Argo, e.g. the Python process in the container locks up), I can get the workflow "unstuck" temporarily, to some degree, by forcing the pod to fail (exec -it, kill python). The step has a retryPolicy. It is retried, and at the same time a few more of those "stuck" branches get started up. But definitely not all of them.
I confirmed this hack to work on 4 of my stuck workflows. All 4 of them had exactly one frozen pod. If a workflow doesn't have any frozen pods, I have no way to manually kick it this way. To be sure, I am not suggesting that frozen containers are causing the workflows to get stuck, this is merely a "lucky" workaround.
@alexec I'm currently on your :grace engineering build as per #4565 testing - that is probably orthogonal, but just mentioning for completeness.
I'm going to try :mot now.

on :mot, with MAX_OPERATION_TIME=30s, confirmed in logs: defaultRequeueTime=30s maxOperationTime=30s
everything just lit up like a christmas tree! :heart_eyes_cat:
all "stuck" steps started up.
@alexec not to jinx it, but i think we have a winner!

Can I play this back:

:mot with 30s fixed your problem.
:grace did not sort fix the problem

Correct!

We were able to replicate the findings here - can verify that the CPU usage is way down with hundreds of concurrent workflows running.

@tomgoren can you confirm which engineering build you ran please?

@alexec:
    Image:         argoproj/workflow-controller:mot
    Image ID:      docker.io/argoproj/workflow-controller@sha256:202d4d7b575043543cbdead5c4fbbf4c3153d7d9a9eef90d47a85ac084a05860

With:
    Environment:
      MAX_OPERATION_TIME:  30s

I've created a POC engineering build that uses S3 to offload and archive workflows to instead of MySQL or Postgres. My hypothesis is that offloading there maybe faster for users running large (5009+ node) workflows, or that archiving maybe more to many users. On top of this, it maybe cheaper for many users. I challenge you to prove me wrong. https://github.com/argoproj/argo/pull/4582

Offloading to S3 would be very interesting and welcome. Will try it when I can! Over the past weekend we ended up running workflows of enormous size, on the :latest build, and highest node count we were able to get to, even with MAX_OPERATION_TIME=600s, was about 50,000 nodes. It's probably unreasonable to expect Argo to handle more at this point, but if anyone was curious about testing the limits, here you go :) (we ended up having to split our workflows into much smaller chunks, for sanity and observability's sake).

50,000 nodes.

Excellent. Can you share a screenshot?

We don't have those workflows in the cluster anymore, and I didn't take a screenshot! I'll try to run one of the larger workflows again over the holidays to take the screenshot

I think this is now fixe.d