[Bug] Queries taking too long

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue
Around about the 2nd August I upgraded (all packages and dbt) but in particular this package from 0.3.0 to 0.4.0
Since then the load on my small PostgreSQL instance has gone from averaging at 11% to 90%

What I see is the query below running for hours
The only tables in this query with significant rows are the territory tables but they are circa 124000 rows only.
I'm running PostgreSQL in Azure. I'm no DB expert so haven't been really able to dig into this.
Happy to debug and help as needed or be told I'm doing it wrong :)
Relevant error log or model output
CREATE TABLE
  "dwh_prod"."dbt_apple_store"."apple_store__territory_report__dbt_tmp"
AS (
WITH
  app AS (SELECT * FROM "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__app"),
  app_store_territory AS (SELECT * FROM "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__app_store_territory"),
  country_codes AS (SELECT * FROM "dwh_prod"."dbt_apple_store_source"."apple_store_country_codes"),
  downloads_territory AS (SELECT * FROM "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__downloads_territory"),
  usage_territory AS (SELECT * FROM "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__usage_territory"),
  reporting_grain AS (SELECT DISTINCT source_relation, date_day, app_id, source_type, territory FROM app_store_territory),
  joined AS (
     SELECT
         reporting_grain.source_relation,
         reporting_grain.date_day,
         reporting_grain.app_id,
         app.app_name,
         reporting_grain.source_type,
         reporting_grain.territory AS territory_long,
         coalesce(official_country_codes.country_code_alpha_2, alternative_country_codes.country_code_alpha_2) AS territory_short,
         coalesce(official_country_codes.region, alternative_country_codes.region) AS region,
         coalesce(official_country_codes.sub_region, alternative_country_codes.sub_region) AS sub_region,
         coalesce(app_store_territory.impressions, 0) AS impressions,
         coalesce(app_store_territory.impressions_unique_device, 0) AS impressions_unique_device,
         coalesce(app_store_territory.page_views, 0) AS page_views,
         coalesce(app_store_territory.page_views_unique_device, 0) AS page_views_unique_device,
         coalesce(downloads_territory.first_time_downloads, 0) AS first_time_downloads,
         coalesce(downloads_territory.redownloads, 0) AS redownloads,
         coalesce(downloads_territory.total_downloads, 0) AS total_downloads,
         coalesce(usage_territory.active_devices, 0) AS active_devices,
         coalesce(usage_territory.active_devices_last_30_days, 0) AS active_devices_last_30_days,
         coalesce(usage_territory.deletions, 0) AS deletions,
         coalesce(usage_territory.installations, 0) AS installations,
         coalesce(usage_territory.sessions, 0) AS sessions
     FROM reporting_grain
     LEFT JOIN app
         on reporting_grain.app_id = app.app_id
         AND reporting_grain.source_relation = app.source_relation
     LEFT JOIN app_store_territory
         on reporting_grain.date_day = app_store_territory.date_day
         AND reporting_grain.source_relation = app_store_territory.source_relation
         AND reporting_grain.app_id = app_store_territory.app_id
         AND reporting_grain.source_type = app_store_territory.source_type
         AND reporting_grain.territory = app_store_territory.territory
     LEFT JOIN downloads_territory
         on reporting_grain.date_day = downloads_territory.date_day
         AND reporting_grain.source_relation = downloads_territory.source_relation
         AND reporting_grain.app_id = downloads_territory.app_id
         AND reporting_grain.source_type = downloads_territory.source_type
         AND reporting_grain.territory = downloads_territory.territory
     LEFT JOIN usage_territory
         on reporting_grain.date_day = usage_territory.date_day
         AND reporting_grain.source_relation = usage_territory.source_relation
         AND reporting_grain.app_id = usage_territory.app_id
         AND reporting_grain.source_type = usage_territory.source_type
         AND reporting_grain.territory = usage_territory.territory
     LEFT JOIN country_codes AS official_country_codes
         on reporting_grain.territory = official_country_codes.country_name
     LEFT JOIN country_codes AS alternative_country_codes
         on reporting_grain.territory = alternative_country_codes.alternative_country_name
 )
 SELECT * FROM joined;

Expected behavior
Query completes within a reasonable time
dbt Project configurations
name: 'gladly'
version: '1.0.0'
config-version: 2

profile: 'gladly'

model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"
clean-targets:
  - "target"
  - "dbt_packages"

vars:

  # Stripe settings
  stripe_timezone: "Australia/Sydney"
  stripe_schema: stripe_load
  stripe__using_invoices: False
  stripe__using_subscriptions: False
  stripe__using_credit_notes: False
  stripe__using_subscription_history: True

  # Play Store
  google_play_schema: google_play_load

  # App Store
  apple_store_schema: itunes_connect_load

  # Amplitude
  amplitude_schema: amplitude_load

  # Xero
  xero_schema: xero_load

  # Twitter Ads
  twitter_ads_schema: twitter_ads_load
  twitter_ads__using_keywords: False # Our account doesn't use keywords

  # Facebook Ads
  facebook_ads_schema: facebook_ads_load

  # Ad Reporting
  ad_reporting__apple_search_ads_enabled: False
  ad_reporting__pinterest_ads_enabled: False
  ad_reporting__microsoft_ads_enabled: False
  ad_reporting__linkedin_ads_enabled: False
  ad_reporting__google_ads_enabled: False
  ad_reporting__snapchat_ads_enabled: False
  ad_reporting__tiktok_ads_enabled: False
  ad_reporting__amazon_ads_enabled: False
  ad_reporting__reddit_ads_enabled: False

  # Linkedin pages
  linkedin_pages_schema: linkedin_company_pages_load

  # Twitter organic
  twitter_organic_schema: twitter_organic_load

  # Youtube analytics
  youtube_analytics_schema: youtube_analytics_load
  youtube__using_video_metadata: true
  youtube_metadata_schema: youtube_analytics_load
  youtube__channel_basic_table: channel_basic_a_2
  youtube__channel_demographics_table: channel_demographics_a_1

  # Facebook Pages
  facebook_pages_schema: facebook_ads_load

  # Instagram Business
  instragram_business_schema: instragram_business_load

  # Dynamo DB Settings
  dynamodb_schema: dynamodb_load
  dynamodb_suffix: qry_4_ki_4_hvzhm_5_bnv_26_qtcqscaq_prod

models:
  stripe_source:
    +schema: stripe_stg

  google_play_source:
    +schema: google_play_stg

  apple_store_source:
    +schema: apple_store_stg

  amplitude_source:
    +schema: amplitude_stg

  xero_source:
    +schema: xero_stg

  twitter_ads_source:
    +schema: twitter_ads_stg

  twitter_organic_source:
    +schema: twitter_organic_stg

  facebook_ads_source:
    +schema: facebook_ads_stg

  linkedin_pages_source:
    +schema: linkedin_company_pages_stg

  youtube_analytics_source:
    +schema: youtube_analytics_stg

  facebook_pages_source:
    +schema: facebook_pages_stg

  instagram_business_source:
    +schema: instagram_business_stg

  gladly:
    dynamodb:
      +schema: gladly
      +materialized: view
    stats:
      +schema: stats
      +materialized: view

Package versions
packages:
  - package: fivetran/stripe
    version: [">=0.14.0", "<0.15.0"]

  - package: fivetran/google_play
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/apple_store
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/app_reporting
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/amplitude
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/xero
    version: [">=0.6.0", "<0.7.0"]

  - package: fivetran/twitter_ads
    version: [">=0.7.0", "<0.8.0"]

  - package: fivetran/facebook_ads
    version: [">=0.7.0", "<0.8.0"]

  - package: fivetran/ad_reporting
    version: [">=1.9.0", "<1.10.0"]

  - package: fivetran/linkedin_pages
    version: [">=0.3.0", "<0.4.0"]

  - package: fivetran/twitter_organic
    version: [">=0.2.0", "<0.3.0"]

  - package: fivetran/youtube_analytics
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/facebook_pages
    version: [">=0.3.0", "<0.4.0"]

  - package: fivetran/instagram_business
    version: [">=0.2.0", "<0.3.0"]

  - package: fivetran/social_media_reporting
    version: [">=0.4.0", "<0.5.0"]

What database are you using dbt with?
postgres
dbt Version
1.7.17 (Note I'm also using dbt transformations, same issue in both places)
Additional Context
No response
Are you willing to open a PR to help address this issue?

[X] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[ ] No.

Hi @johnf thanks for reaching out to the team! Could you confirm the following to help me get a sense of what might be going on:

From your graph, am I understanding this is a consistent issue, or would you say certain runs more problematic and cause the spikes?
I see you have a lot of our packages (awesome!). Could you expand on how you found that apple_store was the culprit? Were you able to run it in isolation by any chance?

Hi @fivetran-catfritz,
I narrowed it down to apple_store because the query I pasted above looks like the problematic one. Right now it's been running for 10 hours.

I'm not sure if you have any internal stats on the transformations, but if you do I suspect you'll see the apple ones are running all the time on my account (johnf@gladlyapp.com).
I'll pause it now so we can confirm 100%

Hi @johnf I reviewed your run logs and noticed the issue you mentioned. Could you try running this package on its own to see if the problem persists?
We've seen similar hangups with Postgres in dbt projects involving multiple packages, often alongside minor updates—like adding a single text field in this case. More puzzling, I've also encountered this issue in another Postgres instance where it occurred in a *_tmp model that was simply a select *.
The issue hasn't appeared consistently enough for us to determine a root cause. I suspect Postgres locks might be a factor, though that’s just a theory for now. Running the apple_store package in isolation could help clarify if that’s the case.

@fivetran-catfritz I can confirm that if I disable the apple_store the load stays at 10% for hours.
I'll try enabling only it to see what happens.
Also worth noting that when that query is running it is the only one running

I've paused all transformations in the dashboard and then commented out all other packages and run them locally.
It's currently hanging on
22:53:15  39 of 44 START sql table model dbt_apple_store.apple_store__territory_report ... [RUN]

I suspect it will sit there for hours again.
FYI, I am happy to jump on a call if someone wants to guide me on running queries manually. This is very reproducible. Also, happy for you to connect to my DB from your end directly.
I kicked it off at 9:30am

With no other queries running, it does finish eventually but takes 80 minutes.
23:29:28  39 of 44 START sql table model dbt_apple_store.apple_store__territory_report ... [RUN]
00:49:08  39 of 44 OK created sql table model dbt_apple_store.apple_store__territory_report  [SELECT 130595 in 4780.70s]
00:49:08  40 of 44 START sql table model dbt_apple_store.apple_store__app_version_report . [RUN]
01:21:40  40 of 44 OK created sql table model dbt_apple_store.apple_store__app_version_report  [SELECT 261984 in 1951.45s]
01:21:40  41 of 44 START sql table model dbt_apple_store.apple_store__device_report ...... [RUN]
01:22:41  41 of 44 OK created sql table model dbt_apple_store.apple_store__device_report . [SELECT 18174 in 61.07s]
01:22:41  42 of 44 START sql table model dbt_apple_store.apple_store__platform_version_report  [RUN]

It isn't a super powerful instance, it is an Azure Burstable, B1ms, 1 vCores, 2 GiB RAM, 32 GiB storage but this seems like a long time for not that much data.

Hi @johnf Thank you for running that! 80 mins does seem a bit long for the size. Before we dive deeper, could you check one more thing? If you downgrade to the prior version of dbt_apple_store and run that independently, what is the run time like? I think it would help to have a couple baselines.

@fivetran-catfritz OK good news, it's super fast on the old version, only 3 seconds.
21:50:16  39 of 44 START sql table model dbt_apple_store.apple_store__territory_report ... [RUN]
21:50:19  39 of 44 OK created sql table model dbt_apple_store.apple_store__territory_report  [SELECT 133045 in 3.33s]

For reference this is the diff between the queries
--- before	2024-08-28 07:54:58.434667925 +1000
+++ after	2024-08-28 07:54:59.798667716 +1000
@@ -23,6 +23,7 @@
  ),
  reporting_grain as (
      select distinct
+         source_relation,
          date_day,
          app_id,
          source_type,
@@ -31,6 +32,7 @@
  ),
  joined as (
      select
+         reporting_grain.source_relation,
          reporting_grain.date_day,
          reporting_grain.app_id,
          app.app_name,
@@ -54,18 +56,22 @@
      from reporting_grain
      left join app
          on reporting_grain.app_id = app.app_id
+         and reporting_grain.source_relation = app.source_relation
      left join app_store_territory
          on reporting_grain.date_day = app_store_territory.date_day
+         and reporting_grain.source_relation = app_store_territory.source_relation
          and reporting_grain.app_id = app_store_territory.app_id
          and reporting_grain.source_type = app_store_territory.source_type
          and reporting_grain.territory = app_store_territory.territory
      left join downloads_territory
          on reporting_grain.date_day = downloads_territory.date_day
+         and reporting_grain.source_relation = downloads_territory.source_relation
          and reporting_grain.app_id = downloads_territory.app_id
          and reporting_grain.source_type = downloads_territory.source_type
          and reporting_grain.territory = downloads_territory.territory
      left join usage_territory
          on reporting_grain.date_day = usage_territory.date_day
+         and reporting_grain.source_relation = usage_territory.source_relation
          and reporting_grain.app_id = usage_territory.app_id
          and reporting_grain.source_type = usage_territory.source_type
          and reporting_grain.territory = usage_territory.territory

I also ran explain on both querys, but t's greek to me :grin:
FAST
 Hash Left Join  (cost=31584.11..37422.28 rows=83150 width=1268)
   Hash Cond: (app_store_territory_1.app_id = stg_apple_store__app.app_id)
   CTE app_store_territory
     ->  Seq Scan on stg_apple_store__app_store_territory  (cost=0.00..3122.45 rows=133045 width=70)
   CTE country_codes
     ->  Seq Scan on apple_store_country_codes  (cost=0.00..6.50 rows=250 width=71)
   ->  Merge Right Join  (cost=28427.16..31303.04 rows=20788 width=2300)
         Merge Cond: ((stg_apple_store__downloads_territory.territory = app_store_territory_1.territory) AND (stg_apple_store__downloads_territory.date_day = app_store_territory_1.date_day) AND (stg_apple_store__downloads_territory.app_id = app_store_territory_1.app_id) AND (stg_apple_store__downloads_territory.source_type = app_store_territory_1.source_type))
         ->  Sort  (cost=17939.56..18273.07 rows=133406 width=61)
               Sort Key: stg_apple_store__downloads_territory.territory, stg_apple_store__downloads_territory.date_day, stg_apple_store__downloads_territory.app_id, stg_apple_store__downloads_territory.source_type
               ->  Seq Scan on stg_apple_store__downloads_territory  (cost=0.00..2998.06 rows=133406 width=61)
         ->  Materialize  (cost=10487.61..11488.02 rows=20788 width=2276)
               ->  Merge Left Join  (cost=10487.61..11436.05 rows=20788 width=2276)
                     Merge Cond: (app_store_territory_1.territory = (alternative_country_codes.alternative_country_name)::text)
                     ->  Merge Left Join  (cost=10472.65..11067.70 rows=16630 width=1212)
                           Merge Cond: (app_store_territory_1.territory = (official_country_codes.country_name)::text)
                           ->  Merge Left Join  (cost=10457.69..10770.03 rows=13304 width=148)
                                 Merge Cond: ((app_store_territory_1.territory = stg_apple_store__usage_territory.territory) AND (app_store_territory_1.date_day = stg_apple_store__usage_territory.date_day) AND (app_store_territory_1.app_id = stg_apple_store__usage_territory.app_id) AND (app_store_territory_1.source_type = stg_apple_store__usage_territory.source_type))
                                 ->  Sort  (cost=9359.64..9392.90 rows=13304 width=108)
                                       Sort Key: app_store_territory_1.territory, app_store_territory_1.date_day, app_store_territory_1.app_id, app_store_territory_1.source_type
                                       ->  Hash Right Join  (cost=4390.47..8448.34 rows=13304 width=108)
                                             Hash Cond: ((app_store_territory.date_day = app_store_territory_1.date_day) AND (app_store_territory.app_id = app_store_territory_1.app_id) AND (app_store_territory.source_type = app_store_territory_1.source_type) AND (app_store_territory.territory = app_store_territory_1.territory))
                                             ->  CTE Scan on app_store_territory  (cost=0.00..2660.90 rows=133045 width=108)
                                             ->  Hash  (cost=4124.39..4124.39 rows=13304 width=76)
                                                   ->  HashAggregate  (cost=3991.35..4124.39 rows=13304 width=76)
                                                         Group Key: app_store_territory_1.date_day, app_store_territory_1.app_id, app_store_territory_1.source_type, app_store_territory_1.territory
                                                         ->  CTE Scan on app_store_territory app_store_territory_1  (cost=0.00..2660.90 rows=133045 width=76)
                                 ->  Sort  (cost=1098.06..1127.26 rows=11682 width=77)
                                       Sort Key: stg_apple_store__usage_territory.territory, stg_apple_store__usage_territory.date_day, stg_apple_store__usage_territory.app_id, stg_apple_store__usage_territory.source_type
                                       ->  Seq Scan on stg_apple_store__usage_territory  (cost=0.00..308.82 rows=11682 width=77)
                           ->  Sort  (cost=14.96..15.58 rows=250 width=1580)
                                 Sort Key: official_country_codes.country_name
                                 ->  CTE Scan on country_codes official_country_codes  (cost=0.00..5.00 rows=250 width=1580)
                     ->  Sort  (cost=14.96..15.58 rows=250 width=1580)
                           Sort Key: alternative_country_codes.alternative_country_name
                           ->  CTE Scan on country_codes alternative_country_codes  (cost=0.00..5.00 rows=250 width=1580)
   ->  Hash  (cost=18.00..18.00 rows=800 width=40)
         ->  Seq Scan on stg_apple_store__app  (cost=0.00..18.00 rows=800 width=40)
(38 rows)

SLOW
 Hash Left Join  (cost=32314.14..36206.23 rows=20788 width=1300)
   Hash Cond: (app_store_territory_1.territory = (alternative_country_codes.alternative_country_name)::text)
   CTE app_store_territory
     ->  Seq Scan on stg_apple_store__app_store_territory  (cost=0.00..3122.45 rows=133045 width=70)
   CTE country_codes
     ->  Seq Scan on apple_store_country_codes  (cost=0.00..6.50 rows=250 width=71)
   ->  Hash Left Join  (cost=29177.07..32300.01 rows=16630 width=1300)
         Hash Cond: (app_store_territory_1.territory = (official_country_codes.country_name)::text)
         ->  Merge Left Join  (cost=29168.94..31676.58 rows=13304 width=236)
               Merge Cond: ((app_store_territory_1.app_id = stg_apple_store__usage_territory.app_id) AND (app_store_territory_1.source_relation = stg_apple_store__usage_territory.source_relation))
               Join Filter: ((app_store_territory_1.date_day = stg_apple_store__usage_territory.date_day) AND (app_store_territory_1.source_type = stg_apple_store__usage_territory.source_type) AND (app_store_territory_1.territory = stg_apple_store__usage_territory.territory))
               ->  Merge Left Join  (cost=28070.88..30356.40 rows=13304 width=196)
                     Merge Cond: ((app_store_territory_1.app_id = stg_apple_store__downloads_territory.app_id) AND (app_store_territory_1.source_relation = stg_apple_store__downloads_territory.source_relation))
                     Join Filter: ((app_store_territory_1.date_day = stg_apple_store__downloads_territory.date_day) AND (app_store_territory_1.source_type = stg_apple_store__downloads_territory.source_type) AND (app_store_territory_1.territory = stg_apple_store__downloads_territory.territory))
                     ->  Merge Left Join  (cost=10131.33..10239.77 rows=13304 width=172)
                           Merge Cond: ((app_store_territory_1.app_id = stg_apple_store__app.app_id) AND (app_store_territory_1.source_relation = stg_apple_store__app.source_relation))
                           ->  Sort  (cost=10074.75..10108.01 rows=13304 width=140)
                                 Sort Key: app_store_territory_1.app_id, app_store_territory_1.source_relation
                                 ->  Hash Right Join  (cost=4756.34..9163.46 rows=13304 width=140)
                                       Hash Cond: ((app_store_territory.date_day = app_store_territory_1.date_day) AND (app_store_territory.source_relation = app_store_territory_1.source_relation) AND (app_store_territory.app_id = app_store_territory_1.app_id) AND (app_store_territory.source_type = app_store_territory_1.source_type) AND (app_store_territory.territory = app_store_territory_1.territory))
                                       ->  CTE Scan on app_store_territory  (cost=0.00..2660.90 rows=133045 width=140)
                                       ->  Hash  (cost=4457.00..4457.00 rows=13304 width=108)
                                             ->  HashAggregate  (cost=4323.96..4457.00 rows=13304 width=108)
                                                   Group Key: app_store_territory_1.source_relation, app_store_territory_1.date_day, app_store_territory_1.app_id, app_store_territory_1.source_type, app_store_territory_1.territory
                                                   ->  CTE Scan on app_store_territory app_store_territory_1  (cost=0.00..2660.90 rows=133045 width=108)
                           ->  Sort  (cost=56.58..58.58 rows=800 width=72)
                                 Sort Key: stg_apple_store__app.app_id, stg_apple_store__app.source_relation
                                 ->  Seq Scan on stg_apple_store__app  (cost=0.00..18.00 rows=800 width=72)
                     ->  Materialize  (cost=17939.56..18606.59 rows=133406 width=62)
                           ->  Sort  (cost=17939.56..18273.07 rows=133406 width=62)
                                 Sort Key: stg_apple_store__downloads_territory.app_id, stg_apple_store__downloads_territory.source_relation
                                 ->  Seq Scan on stg_apple_store__downloads_territory  (cost=0.00..2998.06 rows=133406 width=62)
               ->  Sort  (cost=1098.06..1127.26 rows=11682 width=78)
                     Sort Key: stg_apple_store__usage_territory.app_id, stg_apple_store__usage_territory.source_relation
                     ->  Seq Scan on stg_apple_store__usage_territory  (cost=0.00..308.82 rows=11682 width=78)
         ->  Hash  (cost=5.00..5.00 rows=250 width=1580)
               ->  CTE Scan on country_codes official_country_codes  (cost=0.00..5.00 rows=250 width=1580)
   ->  Hash  (cost=5.00..5.00 rows=250 width=1580)
         ->  CTE Scan on country_codes alternative_country_codes  (cost=0.00..5.00 rows=250 width=1580)
(39 rows)

@johnf Oh wow ok something is going on there. Thank you for confirming that. The only change from v0.3.0 to v0.4.0 is to allow users to union schemas created by multiple connectors together, which is what the source_relation field is used for. Since you don't need it, I would downgrade in the meantime. For now you can install the below app_reporting range and remove the reference to the app_store and google_play packages, and it will automatically install the correct downgraded versions.
  - package: fivetran/app_reporting
    version: [">=0.3.0", "<0.4.0"]

As for the proper fix, let me talk to my team and get some suggestions on what to explore next. Thanks again for your help with this!

Hi @johnf The next thing we'd like to do is try to narrow down what aspect of the udpates is causing the hangup. I have created a test branch that removes the source_relation join conditions added in v0.4.0 while keeping the other updates the same. Could you run this test branch on its own and see if you are still getting the same hangups? You can install this branch by using the below code in place of the apple_store code in your packages.yml.
- git: https://github.com/fivetran/dbt_apple_store.git
  test/remove-source-relation-joins
  warn-unpinned: false

Let me know if you have any questions!

Hi, @fivetran-catfritz, the branch works fine. It took about 4 seconds for the bad query

Very interesting. Thank you @johnf for running the test! It confirms there is definitely something going on with those joins. We'll look into this some more and keep you posted.

Hey @johnf 👋
We've done some investigating here and believe for Postgres the join conditions which result in joining on empty strings ('') could have some adverse effects on Postgres which is what you're seeing.
We have a potential solution which we've applied in the following package version:
packages:
  - git: https://github.com/fivetran/dbt_apple_store.git
    revision: feature/union-data-performance-enhancement
    warn-unpinned: false

You can see the changes coming from this branch here from the source package and here from the transform.

Please note if you end up testing this out you'll probably want to turn off your app reporting dependency since this will cause a dependency issue. Let me know if you have any problems.

If you want, you can install the above package version and run the following command since I only made the relevant changes to the apple_store__territory_report upstream models.
dbt run -s +apple_store__territory_report

Let me know if this helps the performance!

@fivetran-joemarkiewicz
I'm just tested this and I think it has the same problem.
This is the query that ran is below. Still running after 10 minutes
FYI If it's possible on your end then happy for you to connect straight to the DB to test.
I can give you SSH to my proxy server where I'm testing if that helps
query         | /* {"app": "dbt", "dbt_version": "1.7.17", "profile_name": "gladly", "target_name": "prod", "node_id": "model.apple_store.apple_store__territory_report"} */+
              |                                                                                +
              |                                                                                +
              |                                                                                +
              |                                                                                +
              |   create  table "dwh_prod"."dbt_apple_store"."apple_store__territory_report__dbt_tmp"                                                                       +
              |                                                                                +
              |                                                                                +
              |     as                                                                                +
              |                                                                                +
              |   (                                                                                +
              |     with app as (                                                                                +
              |                                                                                +
              |     select *                                                                                +
              |     from "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__app"                                                                                +
              | ),                                                                                +
              |                                                                                +
              | app_store_territory as (                                                                                +
              |                                                                                +
              |     select *                                                                                +
              |     from "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__app_store_territory"                                                                            +
              | ),                                                                                +
              |                                                                                +
              | country_codes as (                                                                                +
              |                                                                                +
              |     select *                                                                                +
              |     from "dwh_prod"."dbt_apple_store_source"."apple_store_country_codes"                                                                                +
              | ),                                                                                +
              |                                                                                +
              | downloads_territory as (                                                                                +
              |                                                                                +
              |     select *                                                                                +
              |     from "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__downloads_territory"                                                                            +
              | ),                                                                                +
              |                                                                                +
              | usage_territory as (                                                                                +
              |                                                                                +
              |     select *                                                                                +
              |     from "dwh_prod"."dbt_apple_store_stg"."stg_apple_store__usage_territory"                                                                                +
              | ),                                                                                +
              |                                                                                +
              | reporting_grain as (                                                                                +
              |                                                                                +
              |     select distinct                                                                                +
              |         source_relation,                                                                                +
              |         date_day,                                                                                +
              |         app_id,                                                                                +
              |         source_type,                                                                                +
              |         territory                                                                                +
              |     from app_store_territory                                                                                +
              | ),                                                                                +
              |                                                                                +
              | joined as (                                                                                +
              |                                                                                +
              |     select                                                                                +
              |         reporting_grain.source_relation,                                                                                +
              |         reporting_grain.date_day,                                                                                +
              |         reporting_grain.app_id,                                                                                +
              |         app.app_name,                                                                                +
              |         reporting_grain.source_type,                                                                                +
              |         reporting_grain.territory as territory_long,                                                                                +
              |         coalesce(official_country_codes.country_code_alpha_2, alternative_country_codes.country_code_alpha_2) as territory_short,                           +
              |         coalesce(official_country_codes.region, alternative_country_codes.region) as region,                                                                +
              |         coalesce(official_country_codes.sub_region, alternative_country_codes.sub_region) as sub_region,                                                    +
              |         coalesce(app_store_territory.impressions, 0) as impressions,                                                                                +
              |         coalesce(app_store_territory.impressions_unique_device, 0) as impressions_unique_device,                                                            +
              |         coalesce(app_store_territory.page_views, 0) as page_views,                                                                                +
              |         coalesce(app_store_territory.page_views_unique_device, 0) as page_views_unique_device,                                                              +
              |         coalesce(downloads_territory.first_time_downloads, 0) as first_time_downloads,                                                                      +
              |         coalesce(downloads_territory.redownloads, 0) as redownloads,                                                                                +
              |         coalesce(downloads_territory.total_downloads, 0) as total_downloads,                                                                                +
              |         coalesce(usage_territory.active_devices, 0) as active_devices,                                                                                +
              |         coalesce(usage_territory.active_devices_last_30_days, 0) as active_devices_last_30_days,                                                            +
              |         coalesce(usage_territory.deletions, 0) as deletions,                                                                                +
              |         coalesce(usage_territory.installations, 0) as installations,                                                                                +
              |         coalesce(usage_territory.sessions, 0) as sessions                                                                                +
              |     from reporting_grain                                                                                +
              |     left join app                                                                                +
              |         on reporting_grain.app_id = app.app_id                                                                                +
              |         and reporting_grain.source_relation = app.source_relation                                                                                +
              |     left join app_store_territory                                                                                +
              |         on reporting_grain.date_day = app_store_territory.date_day                                                                                +
              |         and reporting_grain.source_relation = app_store_territory.source_relation                                                                           +
              |         and reporting_grain.app_id = app_store_territory.app_id                                                                                +
              |         and reporting_grain.source_type = app_store_territory.source_type                                                                                +
              |         and reporting_grain.territory = app_store_territory.territory                                                                                +
              |     left join downloads_territory                                                                                +
              |         on reporting_grain.date_day = downloads_territory.date_day                                                                                +
              |         and reporting_grain.source_relation = downloads_territory.source_relation                                                                           +
              |         and reporting_grain.app_id = downloads_territory.app_id                                                                                +
              |         and reporting_grain.source_type = downloads_territory.source_type                                                                                +
              |         and reporting_grain.territory = downloads_territory.territory                                                                                +
              |     left join usage_territory                                                                                +
              |         on reporting_grain.date_day = usage_territory.date_day                                                                                +
              |         and reporting_grain.source_relation = usage_territory.source_relation                                                                               +
              |         and reporting_grain.app_id = usage_territory.app_id                                                                                +
              |         and reporting_grain.source_type = usage_territory.source_type                                                                                +
              |         and reporting_grain.territory = usage_territory.territory                                                                                +
              |     left join country_codes as official_country_codes                                                                                +
              |         on reporting_grain.territory = official_country_codes.country_name                                                                                +
              |     left join country_codes as alternative_country_codes                                                                                +
              |         on reporting_grain.territory = alternative_country_codes.alternative_country_name                                                                   +
              | )                                                                                +
              |                                                                                +
              | select *                                                                                +
              | from joined                                                                                +
              |   );                                                                                +
              |

Hey @johnf thanks for sharing and sorry that didn't seem to do the trick.
The above was actually my last ditch effort to see if we could avoid the issues we are seeing with constant expressions causing the query planner in Postgres to explode and take significantly longer than necessary. Since the above didn't work, we are going to plan to move forward with the approach where we remove the join condition only if the union schema feature is not being used. As @fivetran-catfritz shared in the branch above, we can confirm removing the join condition when not necessary will address the query performance issue.
We are going to move forward with making this update to the Apple Store and Google Play models. What we envision this update to be is something along the lines of the following whenever we join on source_relation.
from reporting_grain
left join app
      on reporting_grain.app_id = app.app_id
      {{ "and reporting_grain.source_relation = app.source_relation" if var('apple_store_union_schemas',[]) or var('apple_store_union_databases',[]) }}

This way we are excluding the join if the defined variables are empty (not using the union data feature). Ideally, we can consolidate this into a single macro that can be referenced in each model so the code is dryer, but this is what we will likely apply to these models and others going forward.

@fivetran-joemarkiewicz sounds great. Let me know if you need any more testing

Hi @johnf, I wanted to post back because we have been able to spend some time thinking through this issue/solution and we don't feel the solution initially proposed (removing the join condition for non union_data environments) is scalable. This feature is applied to a large number of our existing dbt packages, and making this config change to every join is not something which will scale appropriately.
We have a few other ideas:

Modify the source_relation field to be a composite key to ensure it isn't a constant expression and therefore causing performance issues within the query planner.
Make an adjustment upstream to influence the query planner to not identify the source_relation field as a constant expression.

Unfortunately, our test Postgres instance isn't representing a live environment and it's difficult to validate if the changes we are testing will properly address the issue. I recall you were open to sharing your Postgres instance to help with this bug fix. Before then, would you by chance be available for a 30 minute video call to explore this further and allow us to test our assumptions with you? Let me know if you would be interested. Thanks!

@fivetran-joemarkiewicz Sure happy to help! I just connected with you on LinkedIn so we can organise a time

Thanks so much for meeting with me yesterday @johnf to troubleshoot this issue!
Following our meeting I believe we came to an understanding of where the issue was truly originating. Additionally, I feel I found a viable and appropriate solution. If you would like, you can attempt to run the troublesome model using the below package version.
- git: https://github.com/fivetran/dbt_apple_store.git
  revision: bugfix/postgres-performance-updates
  warn-unpinned: false

A short summary of what the issue was and why I believe the above changes will solve the issue:

The issue was found to be with this exact join (likely an issue in the other end models as well).
The issue with this join was the fact that your upstream app source table only had one record (therefore only 1 app_id). This then resulted in this join essentially being unnecessary since both source_relation and app_id in your case are all constant expressions (the same value for all records). This was not an issue before (pre source_relation) because Postgres is able to handle the load to address one constant expression in a join. However, once we add two in this case it seems to become overloaded.
Additionally, since the report_grain is just a cte in this scenario, it means Postgres must scan over the tables and perform the constant expression joins all in memory (therefore putting stress on the CPU like you were seeing in your screenshots).
As a result of the above, I attempted to move the report grain operation to its own model and materialize it as a table. This still is not 100% performant because Postgres still needs to join two constant expressions in your case. However, it now is able to do this much more efficiently because it can simply scan the table and go from there as opposed to needing to do all of this in memory via a cte.

Ideally, we would just not include the joins at all in your scenario. However, for other customers the app_id and source_relation fields are not constant expressions and therefore do need to be included. As a result, I believe the above approach is a viable, appropriate, and scalable solution which will be relevant to all users of this package. Additionally, this is a great learning for how we should be structuring our intermediate models when dealing with possible constant expression joins in other packages.
I was able to reproduce the error you are seeing and also see this branch resolve the error. On my end I was able to get the territory report to go from never finishing, to completing in just 10 seconds (which seems similar to what you were experiencing in the past). I made similar updates to all the other end models so you should see this issue resolved across this package. Please let me know if this does the trick! If not, we can explore other options to address this performance issue.
Thanks again!

@fivetran-joemarkiewicz Unfortunately this didn't help.
But I discovered something interesting. If I run the dbt command it hangs. If I grab the exact same query from --debug and run it manually from psql it finishes in < 30 seconds.
Does dbt do anything special when it's talking to postgres?

I'm sorry to hear @johnf this didn't resolve your issue 😞
Can you confirm that when running the new package version, do you see any new tables materialized in your destination? The new version should have created a these new tables which was intended to help with the query performance. If those tables weren't created then there may be something in the config that needs to be adjusted so they can materialize and you can benefit from the performance improvement.
I don't believe dbt does anything on the backend other than what you see in the target/run folder 🤔. I did encounter an issue when testing where when I ran the original broken version right before running the new version, it did slow down my dbt job execution time. I essentially needed to let the cpu usage level out before attempting the newer version, and then it would work without issue each subsequent run on the new version. I'm not sure if this is the same sequence you took, but wanted to share in case it was.

@johnf just wanted to bump my previous message.

@fivetran-joemarkiewicz Thanks for the reminder. I just cleared out dbt_packages and retried and it works!!
Maybe a caching issue in packages or psql last time??
21:49:47  Found 60 models, 1 seed, 20 tests, 22 sources, 0 exposures, 0 metrics, 650 macros, 0 groups, 0 semantic models
21:49:47
21:49:48  Concurrency: 1 threads (target='prod')
21:49:48
21:49:48  1 of 10 START sql view model dbt_apple_store_stg.stg_apple_store__app_store_territory_tmp  [RUN]
21:49:48  1 of 10 OK created sql view model dbt_apple_store_stg.stg_apple_store__app_store_territory_tmp  [CREATE VIEW in 0.20s]
21:49:48  2 of 10 START sql view model dbt_apple_store_stg.stg_apple_store__app_tmp ...... [RUN]
21:49:48  2 of 10 OK created sql view model dbt_apple_store_stg.stg_apple_store__app_tmp . [CREATE VIEW in 0.07s]
21:49:48  3 of 10 START sql view model dbt_apple_store_stg.stg_apple_store__downloads_territory_tmp  [RUN]
21:49:48  3 of 10 OK created sql view model dbt_apple_store_stg.stg_apple_store__downloads_territory_tmp  [CREATE VIEW in 0.07s]
21:49:48  4 of 10 START sql view model dbt_apple_store_stg.stg_apple_store__usage_territory_tmp  [RUN]
21:49:48  4 of 10 OK created sql view model dbt_apple_store_stg.stg_apple_store__usage_territory_tmp  [CREATE VIEW in 0.07s]
21:49:48  5 of 10 START sql table model dbt_apple_store_stg.stg_apple_store__app_store_territory  [RUN]
21:49:50  5 of 10 OK created sql table model dbt_apple_store_stg.stg_apple_store__app_store_territory  [SELECT 196745 in 1.41s]
21:49:50  6 of 10 START sql table model dbt_apple_store_stg.stg_apple_store__app ......... [RUN]
21:49:50  6 of 10 OK created sql table model dbt_apple_store_stg.stg_apple_store__app .... [SELECT 1 in 0.10s]
21:49:50  7 of 10 START sql table model dbt_apple_store_stg.stg_apple_store__downloads_territory  [RUN]
21:49:52  7 of 10 OK created sql table model dbt_apple_store_stg.stg_apple_store__downloads_territory  [SELECT 197106 in 1.84s]
21:49:52  8 of 10 START sql table model dbt_apple_store_stg.stg_apple_store__usage_territory  [RUN]
21:49:52  8 of 10 OK created sql table model dbt_apple_store_stg.stg_apple_store__usage_territory  [SELECT 12731 in 0.14s]
21:49:52  9 of 10 START sql table model dbt_apple_store.int_apple_store__territory_report_grain  [RUN]
21:49:53  9 of 10 OK created sql table model dbt_apple_store.int_apple_store__territory_report_grain  [SELECT 196745 in 1.02s]
21:49:53  10 of 10 START sql table model dbt_apple_store.apple_store__territory_report ... [RUN]
21:50:02  10 of 10 OK created sql table model dbt_apple_store.apple_store__territory_report  [SELECT 196745 in 9.52s]
21:50:02
21:50:02  Finished running 4 view models, 6 table models in 0 hours 0 minutes and 14.95 seconds (14.95s).
21:50:02
21:50:02  Completed successfully
21:50:02
21:50:02  Done. PASS=10 WARN=0 ERROR=0 SKIP=0 TOTAL=10

That's great news and thank you for confirming! We will be moving forward with this change in the current sprint.

Hi @johnf thanks again for all your help with this issue. However, last week there was a change to the Apple Store Fivetran connector which is resulting in the majority of the reports this dbt package references are no longer syncing up to date data.
The connector now has a new version which you can switch to in order to obtain the same information. However, this new connector is using a different schema from the one this package is built off. Therefore, we will need to modify this package to support the new schema. You can see more details within Issue #28 but you should have also received an email last Friday detailing the changes.
The above being said, we will be prioritize updating the dbt package to support the new schema. Fortunately, we have great learnings from this thread which we will apply to the updates in the coming weeks. I'll be sure to keep you in the loop once the updates are applied so you can confirm the same performance gains we achieved in the previously shared branch are retained within the coming update.
Thank you and we really appreciate your patience.


Update Cocoapods to new version

Hey @lucoceano, I need you to update your Trunk with my account:
CocoaPods - Adding other people as contributors

@lfarah sorry man, took me forever to see this! I've just added.

It's ok.... 😄


Empty allocation dropdown on server creation page with Chrome

Panel
Version of Panel/Daemon: 952d8d154c21ff525ec4f44751e8b34352e00cb6 (develop)
Server's OS: Ubuntu 14.04.5 LTS x64
Your Computer's OS & Browser: Windows 10 x64 Chrome

EDIT: i tested with firefox and it works, its a Chrome issue
Empty allocation dropdown on server creation page, and yes i correctly added allocations
Thanks in advance :)

https://github.com/Pterodactyl/Panel/issues/750

Is this fix pushed to develop ? Because on the latest develop branch commit the bug is present.

Clear your cache. If it works fine in one browser and not in another thats generally the problem.


Automated cherry pick of #7994: ctyun sync secgroup fix & cloudaccount delete fix

Cherry pick of #7994 on release/3.3.
#7994: ctyun sync secgroup fix & cloudaccount delete fix

/lgtm
/approve


Having trouble self hosting.

I successfully set the secrets and published to Cloudflare Workers, invited the bot, waited more than an hour but the slash commands don't show up or work. I'm not sure if it's the worker not taking requests or the bot just not working.

you need to register commands yourself apart from the codebase.
https://discord.com/developers/docs/interactions/application-commands#registering-a-command


remove nix warning and edit readme

This removes the nix warning that showed up when you run the ui with yarn
Also updates the readme to remove any nix related info

@EthnTuttle  @okjodom kindly review


Using RayGUI with Existing GLFW Context

Hi. I saw raygui and it looks pretty cool but I already have a game running using GLFW/GLAD. Is there a way I can use the window with RayGUI without having to create it through the raylib API?

@IndianBoy6 raygui can be used as standalone if required functions are provided by user, just check RAYGUI_STANDALONE define for required functions. Check examples/standalone/raygui_standalone.c for some reference.


fix(image-stream): Remove inability to fallback to octet-stream

Using mime-types in that place made it impossible to use other
other file extensions like .sdcard and have them treated as application/octet-stream
when drag-and-dropping images into Etcher.
This moves the fallback logic in lib/image-stream to .getFromFilePath(),
to still facilitate proper MIME type detection, while allowing it to fall back
to the octet-stream handler, if there's no available handler for a specific type.
Change-Type: patch
Changelog-Entry: Fix not treating unknown images as octet-stream

Can we add new tests that cover this new behavior?

Can we add new tests that cover this new behavior?

Yes, added a new test file and test for an unrecognized extension :)

Is the random.rpi-sdcard file that this PR adds actually used by any of the unit-tests? (perhaps it is, but I can't tell from the Travis logs)
And it looks like we need to add *.rpi-sdcard to the list of extensions that codespell should ignore?

I sent a PR to ignore the extension. The image file is used by this test: https://github.com/resin-io/etcher/blob/master/tests/image-stream/utils.spec.js#L86

Doesn't that test just operate on strings though, and so not actually need a physical file?

@lurch utils.getArchiveMimeType() will do I/O to figure out the type if it doesn't match known stuff via mime.lookup() – which is the case with .rpi-sdcard (because it's not a registered extension) – so needs a corresponding file.

Ah, didn't realise that. Thanks for the clarification :+1:


API cleanup (pre-1.0.0)

Clean up public APIs so they can be frozen for 1.0.0.
This includes removal of unused methods and moving internal methods out of public classes; this makes the public/internal boundary clearer and is less likely to cause code breakages for API consumers in future.

Done in pull request #266


improve yurtstaticset template metadata

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind bug
/kind documentation
/kind enhancement
/kind good-first-issue
/kind feature
/kind question
/kind design
/sig ai
/sig iot
/sig network
/sig storage

/kind feature
What this PR does / why we need it:
If user defines different namespace in pod template with yurtstaticset namespace, yurtstaticset controller don't know how to handle this case.
Which issue(s) this PR fixes:

Fixes #1473
Special notes for your reviewer:

/assign @rambohe-ch
Does this PR introduce a user-facing change?

other Note

/lgtm

/backport release-v1.3


Calling play() on fresh paused tween doesn't play it

Version

Phaser Version: Phaser v3.60.0 Beta 4

Operating system: Windows 10 & iPad OS 15.3

Description

The docs for Tween.play() say "You only need to call this method if you have configured the tween to be paused on creation."
However as demonstrated by the standalone test case below, calling Tween.play() immediately after it was added with scene.tweens.add doesn't actually play the Tween.
Example Test Code

const config = {
  width: 800,
  height: 600,
  scene: {
    create,
  },
};
const game = new Phaser.Game(config);

function create() {

  const circle = this.add.circle(100,100,20, 0xffffff);
  const tween = this.tweens.add({
    targets: circle,
    alpha: 0,
    duration: 1000,
    paused: true,
  })
  tween.play()
}

Expected result: circle fades away
Actual result: circle doesn't fade away
Additional Information
#5454 might have something to do with it
Edit: Locally undoing the 5454 commit, fixes the issue so the tween plays correctly

Also having this issue.

+1 on this issue!

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.


Unable to install onnx 1.4.1 in the virtual environment on Jetson AGX Xavier

System Details

Jetson AGX Xavier
Jetpack - 4.6
Ubuntu - 18.04
Virtual environment - venv
pip version  - 21.3.1

I am unable to install the onnx version 1.4.1 as mentioned in this repository in the virtual environment. However, when I do pip install onnx , it installs the new version of onnx without any problem. Could you please check.
(jkjung_yolo_1) user@user-desktop:~$ pip install onnx==1.4.1
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting onnx==1.4.1
  Downloading onnx-1.4.1.tar.gz (2.9 MB)
     |████████████████████████████████| 2.9 MB 4.9 MB/s
  Preparing metadata (setup.py) ... done
Requirement already satisfied: protobuf in /usr/local/lib/python3.6/dist-packages (from onnx==1.4.1) (3.19.4)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from onnx==1.4.1) (1.19.4)
Requirement already satisfied: six in ./virtual_env/jkjung_yolo_1/lib/python3.6/site-packages (from onnx==1.4.1) (1.16.0)
Requirement already satisfied: typing>=3.6.4 in ./virtual_env/jkjung_yolo_1/lib/python3.6/site-packages (from onnx==1.4.1) (<IP_ADDRESS>)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in ./virtual_env/jkjung_yolo_1/lib/python3.6/site-packages (from onnx==1.4.1) (4.1.1)
Using legacy 'setup.py install' for onnx, since package 'wheel' is not installed.
Installing collected packages: onnx
    Running setup.py install for onnx ... error
    ERROR: Command errored out with exit status 1:
     command: /home/user/virtual_env/jkjung_yolo_1/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_g5i54n9/onnx_0d66111426534ee5b5c48c7a0d7063c8/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_g5i54n9/onnx_0d66111426534ee5b5c48c7a0d7063c8/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-j4v5fy6h/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/virtual_env/jkjung_yolo_1/include/site/python3.6/onnx
         cwd: /tmp/pip-install-_g5i54n9/onnx_0d66111426534ee5b5c48c7a0d7063c8/
    Complete output (65 lines):
    fatal: not a git repository (or any of the parent directories): .git
    WARNING: The wheel package is not available.
    running install
    running build
    running build_py
    running create_version
    running cmake_build
    -- Build type not set - defaulting to Release
    -- The C compiler identification is GNU 7.5.0
    -- The CXX compiler identification is GNU 7.5.0
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working C compiler: /usr/bin/cc - skipped
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /usr/bin/c++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    CMake Error at CMakeLists.txt:217 (message):
      Protobuf compiler not found
    Call Stack (most recent call first):
      CMakeLists.txt:248 (relative_protobuf_generate_cpp)

    -- Configuring incomplete, errors occurred!
    See also "/tmp/pip-install-_g5i54n9/onnx_0d66111426534ee5b5c48c7a0d7063c8/.setuptools-cmake-build/CMakeFiles/CMakeOutput.log".

    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/user/virtual_env/jkjung_yolo_1/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_g5i54n9/onnx_0d66111426534ee5b5c48c7a0d7063c8/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_g5i54n9/onnx_0d66111426534ee5b5c48c7a0d7063c8/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-j4v5fy6h/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/virtual_env/jkjung_yolo_1/include/site/python3.6/onnx Check the logs for full command output.

Apologies, looks like my pip is broken somehow. I checked on my other jetson AGX board and it worked.

https://github.com/jkjung-avt/tensorrt_demos/issues/539#issuecomment-1119218462


Document release process

Fixes #527

Not sure is this good place to document the flow or should it be elsewhere or name the file differently


Crop doesn't hide even if keepSelection set to false

I tried to create a toggle button of crop mode.
When I select an area and click outside on the toggle button I disable the crop, but the selection is still active. Is there anyway to workaround this?

I think you are confused about the behaviour of these two options:
keepSelection - this stops the crop from disappearing if the user clicks outside the crop (somewhere on the image)
disabled - this stops the user from being able to modify the crop, but it doesn't hide or destroy the crop
If you want to temporarily hide it then I recommend using css, try this:
.ReactCrop--crop-selection {
   display: none;
}
.ReactCrop--image-copy {
    -webkit-clip-path: none;
    clip-path: none;
}


fix Number components state getting published with the wrong value #450

fix #450

Wow. Good catch!


Do we need a 2.99.0 release to allow prep for 3.0?

IMO, no, we already provide sufficient deprecations.

I think if we are being proactive about deprecations there's no need for a .99 release series.


Make center of mass computation of a molecule more robust?

Currently, the center of mass computation does not check if all particles of a molecule are adjacent, or if some particles are wrapped into the simulation cell. If a molecule is partially wrapped, the current center of mass computation will yield wrong results.
# Awesome sketch of a partially wrapped molecule
---------------
|        o    |<- boundary of the cell
|     x   \   |
|-o        o -|
---------------
# x : wrong center of mass

In the current implementation, everything is working as long as the initial configuration does not contain "partially wrapped" molecules because there are no propagation methods or other system changes that apply such partial wrapping.
We could enhance the current implementation to check for partial wrapping - e.g. compute bonding lengths without period boundary conditions. Do you think that this is needed? It would make the usage of the existing method more robust with respect to custom extensions.

because there are no propagation methods or other system changes that apply such partial wrapping.

Not yet, but that what #3 is about. So we should handle it before implementing rewrap algorithm.
I think we definitively want to get the "right" center of mass here, I don't know how we can do this.
 Wrap this particular atom?

Just though of an easier solution: using the wrapped distances from the first atom of the molecule:

Because we are wrapping distances from another atom in the molecule, all wrapped values should fall around the same point, and not in separated parts of the cell. What do you think?

For the internal representation of positions, I would favor to never "partially wrap" molecules. I would wrap the molecule as a whole so that its center of mass lies within the cell. That is done at the moment in the translation of molecules  with the implemented wrapping function (again, assuming that all particles of a molecule are adjacent).

What would we do if we find a bonding length without PBC that is bigger than the bonding length wit PBC? Wrap this particular atom?

Or "rewrap" it back so that its real positions is adjacent if we want to keep particles of molecules together.

Because we are wrapping distances from another atom in the molecule, all wrapped values should fall around the same point, and not in separated parts of the cell. What do you think?

That could work. I have think about it 😄

See https://en.wikipedia.org/wiki/Center_of_mass#Systems_with_periodic_boundary_conditions for an algorithm that should always be valid


Fix for command parsing (#45)

I think this is a way to fix #45
Possible command messages for as far as I can see:
/command
/command arg [..] arg
/command@botname
/command@botname arg [..] arg
Instead of splitting the 'command' on "@" or space you can use a regexp to split in one time in three possible parts:

Command (e.g. /image)
Target (e.g. @botname)
arguments (can be multiple) (e.g. 'hello there')

After that nullify the parts that are matched as empty string and pass everything as self.emit. Note: I've added a fourth argument 'target' to the emit call, passing the @botname to the listening application. The application should handle the target if required (match to botname, or do something else like aim a command to a user, etc).
Hope you like the fix :)

I'm not sure about the target part of your code. The bot object from the library represents a single bot, with a unique username, so I don't see the reason it should respond to any command that has arbitrary targets. I think the bot should only respond to those commands that have its own username as target, and ignore the rest.

I agree completely. Only according to the telegram docs the target can also be a user, I think e.g. "/message@user hello". That would be better handed in the app code I think. We could also ignore it (I don't need/use it) but other users of your library might?
At the moment there's no differtation between mentioned users/targets, resulting in the @part to be parsed as command.

I've just tested my bot on the "@part", whenever I'm in private/normal or groupchat with my bot it receives/reacts to all /commands regardsless of an @ . E.g. /image@otherbotname also triggers a response.
At first my bot had privacy mode disabled, but enabling it just now and results seem the same.
So now I'm not completely sure what's the point of the @ part as the bot seem to get all the messages anyhow. Do you have experience with multiple bots in one channel/group?

Thanks. I shall merge first for testing.


Question: Option to deploy with clusterIp (not headless)

We have a tenant using AMQ Broker, but the service is only balancing the connections between 3 of the replicas. How can I set the operator to allow loadbalancing between all AMQ Broker pods? We tried replacing a headless service with clusterIp and it seemed to randomize the connections sufficiently. Is there an option for this or another way to ensure we are loadbalancing evenly? Thanks.

Hi, currently the operator doesn't support this. You need to install your own loadbalancer.


Send Var (var) with Data (val) does not work with a new line

In new line I mean https://scratch.mit.edu/projects/395084280/. This project shown is a hack of having a new line in Scratch, something you can't normally do. I wanted to use this for a project I was working on, but then after testing it with the Send Var blocks it loads the string but not anything after the new line. Please fix the bug :)

Sorry, but it looks like this is a design flaw that would require a radical code rewrite. Packets are formatted using newline characters, and changing how packets are parsed would take too long to patch. Labeling this as a wontfix.

Sorry, but it looks like this is a design flaw that would require a radical code rewrite. Packets are formatted using newline characters, and changing how packets are parsed would take too long to patch. Labeling this as a wontfix.


Find without limit in loop causes process out of memory

I've written a simple test script to compare performance between massive.js and sequelize (which massive was doing incredibly well in until this!) that does a find without a limit, 100 times. I'm doing a find on a table with ~200,000 records in it and after the 14th loop (every time) it throws
<--- Last few GCs --->

   27791 ms: Scavenge 1410.8 (1457.0) -> 1410.8 (1457.0) MB, 16.9 / 0 ms (+ 1.8 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
   28618 ms: Mark-sweep 1410.8 (1457.0) -> 1409.9 (1457.0) MB, 827.7 / 0 ms (+ 2.8 ms in 2 steps since start of marking, biggest step 1.8 ms) [last resort gc].
   29426 ms: Mark-sweep 1409.9 (1457.0) -> 1409.8 (1457.0) MB, 807.1 / 0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x76c271b4629 <JS Object>
    1: parseDate [/Users/jwhitmarsh/src/massive-test/node_modules/massive/node_modules/pg/node_modules/pg-types/node_modules/postgres-date/index.js:~8] [pc=0x32f6cf6679b] (this=0x3db47f92c7a1 <JS Array[9]>,isoDate=0x2078d2b89419 <String[26]: 2015-12-26 19:04:44.406+00>)
    2: new constructor [0x76c271041b9 <undefined>:~1] [pc=0x32f6d00d503] (this=0x2078d2b89471 <JS Object>,parsers=0x3db47f92c7a...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

Obviously this is not a real world scenario so I wouldn't make it a priority! but I thought it might be useful if it's pointing to some memory leakage somewhere?

It would be helpful to actually see your code...

That came out a little shorter than I intended - sorry in the middle of a few things :).
So, if I understand the issue here: you're doing a find without limit, so pulling 200,000 records into memory and you're running out of memory. You didn't say explicitly but I'm guessing Sequelize didn't cause the error?
I haven't looked at their code but I'll guess it's because they utilize streaming for larger queries - which is a good idea always for anything over a few hundred results. You can have Massive stream things by adding {stream: true} to your query options:
db.users.find({company_id : 12}, {stream:true}, function(err,stream){

  stream.on('readable', function(){
    var user = stream.read();
    //do your thing
  });

  stream.on('end', function(){
    //deal with results here
  });
});

This will keep your connection open longer and could use up the pool (depending on what you're doing). Give this a try and see what happens.
Finally, if there's a memory issue it's likely at the driver level. We close off every connection after execution:
https://github.com/robconery/massive-js/blob/master/lib/runner.js#L40
We used to have pooling enabled here but it seemed... buggy - the release never happened as we expected so we just went back to straight querying and it hasn't really caused issues.

No worries, I wasn't sure if the code would be useful or not (since it's just a simple async loop):
function maGetAll(i, next) {
  console.time('maGetAll' + i);
  db.apps.find({}, function(err) {
    if (err) {
      return fail(err);
    }
    console.timeEnd('maGetAll' + i);
    next();
  });
}

function many() {
  async.timesSeries(100, function(i, next) {
    //szCreate(i, next);
    //maCreate(i, next);

    //szGetAll(i, next);
    maGetAll(i, next);
  }, function() {
    process.exit();
  });
}

I'm never actually doing anything with the results so presumably it's being held in memory somewhere, i'm just not sure where...
But it sounds like it's more of an issue with my loop rather than the library?

Well, hard to say really. Pulling 200K records repeatedly and seeing disastrous results really isn't surprising - as you can imagine Massive can't handle memory for you (short of making sure the connection is killed).
Give stream a try.


Meshery Catalog crashing

Description

Enviroment:

OS: iOS
Browser: Safari AND Chrome
Device: Mobile

Contributor Guides and Handbook
The meshery.io website uses Jekyll and GitHub Pages. Site content is found under the master branch.

📚 See the Contributing to Meshery.io Website section of the readme.md and other contributing instructions, too.
🎨 Wireframes and designs for Meshery UI in Figma (open invite)
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

Hi @leecalcote
Is this still an issue, i checked it on both safari and chrome its just working fine.

@ShivamPandey00 thank you for confirming. I think we're good to go here now. 👍


导入自定义文件时出现ModuleNotFoundError: No module named 'dataset_process'的问题

Describe the issue:
在使用torch的dataset自定义数据集时，由于直接在同文件内自定义类时会报错，选择单独生成一个python文件，内部完成数据集的定义，并引入该文件完成，但是在引入自定义文件dataset_process时，程序报错

Environment:

NNI version: v2.8
Training service (local|remote|pai|aml|etc):local|
Client OS:windows11
Server OS (for remote mode only):
Python version:3.6
PyTorch/TensorFlow version:1.7.1
Is conda/virtualenv/venv used?:conda is used
Is running in Docker?:no

Configuration:

Experiment config (remember to remove secrets!):

Search space:

Log message:

nnimanager.log:
nnimanager.log
dispatcher.log:
dispatcher.log
nnictl stdout and stderr:

How to reproduce it?:
使用retiarii在pytorch框架下，引入包含自定义数据集类的代码，如下：

并在nni主程序中import该文件即可报错

What is your trial code directory? Where did you run the trial? Did you use any tricks like sys.path.insert to manipulate Python PATH?

Trial code directory was in my desktop and I simply opened and run it in pycharm. I didn’t use the tricks you mentioned. I would give it a try. Thanks for the advice!

Did you use any tricks like sys.path.insert to manipulate Python PATH?

I mean, please don't use them.
Could you try to run your program in command line like python xxx.py? I'm not sure what pytorch is doing to your code. What is the working directory and etc.

I try to rerun it but get the same problem. This is the working directory and pytorch is 1.7.1 version. Could it be a little old?

I don't think it's related to pytorch version.
Can you import dataset_process in your current working directory?


[REQUEST] POS Create Invoice

You have the option of pulling up sales orders and invoices and paying them using pos_invoice. However we are looking for the inverse options. We would like an option to bill a customer's account (especially if they are net30, etc) instead of paying immediately. ie. A customer comes into the store, and picks up some items for their business. We can ring them up in POS, and their account is billed instead of them being forced to pay immediately or us jumping around making invoices. Ideally a button on the payment options that invoices their account instead of cash/credit.
For use with Version 11

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@gregimagines
As I understand you would like to have an option to create invoice(s) for selected customer from POS payment screen, right?
The flow maybe as follows:

select a customer and add the items in POS
go to payment screen
click, for example, new Create Invoice button
once the button is clicked, new draft invoice is created in backend

Please confirm

That is exactly what I’m looking for, yes. Once button is clicked, invoice is created in backend and POS can go to the next order without having to take a payment on the invoiced one.

@gregimagines
Ok, I've got it and we could help you, but it should be noted that we can't make it for free. Please email to<EMAIL_ADDRESS>to receive a quotation for this customization.


Optimize Paeth decoding

Possible alternative to #377
I was experimenting with the Paeth filter and it seems like the current version works great for encoding but not for decoding for some reason. Using filter_paeth_decode from this PR gave a considerable speed increase but using it for encoding degrades encoding performance. Another slight speed increase came from removing the chunks_exact implementation and iterator juggling in unfilter for Paeth.
I saw ~30% increase in mps decoding speed across several folders. The current status of #377 lands me at 52 mps for decoding.
$ cargo run --release --example corpus-bench -- ./tests/benches/
                Decode
         --------------------
master    46 mps   0.06 GiB/s
f7ad67    57 mps   0.07 GiB/s
295130    61 mps   0.07 GiB/s

I think the distinction is probably that the current filter_paeth works extremely well if the code can be autovectorized, but is slower if not.


Bump Dependencies #8802692494

Bumps dependencies for product-is. Link : https://github.com/wso2/product-is/actions/runs/8802692494

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix(deps): update dependency @dcos/ui-kit to v3.62.0

This PR contains the following updates:

Package
Type
Update
Change

@dcos/ui-kit
dependencies
minor
3.39.0 -> 3.62.0

Release Notes

dcos-labs/ui-kit
v3.62.0
Compare Source
Bug Fixes

name and margin (bfe13cb)

Features

optional icon for selectinput (03894cf)

v3.61.0
Compare Source
Bug Fixes

addresses fieldlist/fieldgroup PR feedback (af28821)
passes 'other' props to link buttons (e8dadbb)

Features

adds FieldGroup component (1b33d5b)
adds FieldList components (8f777b7)

v3.60.1
Compare Source
Bug Fixes

export formStructure components from packages root (fea391b)

v3.60.0
Compare Source
Features

adds top-level form structure components (6ddc708)

v3.59.2
Compare Source
Bug Fixes

toggleboxgroup: makes toggleboxes the same height (573034b)

v3.59.1
Compare Source
Bug Fixes

loadingindicators: adds data attribute for Cypress selectors (38d7e32)

v3.59.0
Compare Source
Features

adds components and docs for loading indicators (1971c8e)

v3.58.0
Compare Source
Features

adds empty state components and documentation (76746e2)

v3.57.0
Compare Source
Features

adds InlineBorderedItems component (b0aff52)

v3.56.0
Compare Source
Features

enables ResetButton component usage (7b532e3)

v3.55.0
Compare Source
Features

textinputwithbadges: adds input value as badge on blur (c6504ea)

v3.54.2
Compare Source
Bug Fixes

codesnippet: enables new lines to be created (d69e584)

v3.54.1
Compare Source
Bug Fixes

arrow direction / asc & desc (26d923b)

v3.54.0
Compare Source
Bug Fixes

improves handling of long unbroken strings in infobox (ffa2f85)

Features

textinputwithbuttons: adds components for text inputs w/ buttons (c99a216)

v3.53.4
Compare Source
Bug Fixes

table cell capitalization (f05039f)

v3.53.3
Compare Source
Bug Fixes

shows hint content under the select input (812f0f7)

v3.53.2
Compare Source
Bug Fixes

scrollable table (bf0557b)

v3.53.1
Compare Source
Bug Fixes

make sidebar scrollable (cfd8431)

v3.53.0
Compare Source
Features

enable lowercase headercell (853edd1)

v3.52.0
Compare Source
Features

adds FullscreenView and refactors FullscreenModal (dcb6fef)

v3.51.0
Compare Source
Features

spacingbox: support different spacing sizes per side (2ee6053)
tabs: adds support for vertical tabs (1e89351)
tabs: makes tab direction responsive (922bfab)

v3.50.1
Compare Source
Bug Fixes

table: uses a workaround to address a react-draggable bug (12d3e8e)

v3.50.0
Compare Source
Features

popoverlistitem: styles disabled items + select handler not called (38465ec)

v3.49.2
Compare Source
Bug Fixes

dropdownable: does not set position if dropdownable is not open (db937bf)

v3.49.1
Compare Source
Bug Fixes

pageheader height (17f60a7)
pageheader: only fills height when tabs are present (985cc68)

v3.49.0
Compare Source
Bug Fixes

table: rounding error in col width calculation (77ad921)

Features

tableview: adds a helper component for table layouts (9b45cb0)

v3.48.0
Compare Source
Features

toggleinputlist: passes last selected item as 2nd onChange param (f756d58)

v3.47.0
Compare Source
Features

modal: allows modals to choose which node they mount in (c95e96d)

v3.46.0
Compare Source
Features

ToggleListInput: add required and labelAppearance (83bd61c)

v3.45.3
Compare Source
Bug Fixes

tabs: fix tab margin (de07f8f)

v3.45.2
Compare Source
Bug Fixes

badge: change line height to 20px (92a65d0)

v3.45.1
Compare Source
Bug Fixes

sidebarsubmenuitem: hides focus outline (4ccf52d)

v3.45.0
Compare Source
Features

adds subtle border around avatar (6b3eb96)

v3.44.0
Compare Source
Features

sidebar: allows sidebar items to be disabled (db12f3d)

v3.43.6
Compare Source
Bug Fixes

makes avatar display block (9f42109)

v3.43.5
Compare Source
Bug Fixes

exports textinputwithbadges from index (ae462fd)

v3.43.4
Compare Source
Bug Fixes

makes it easier to render a Table in a tab panel (5ec14c0)

v3.43.3
Compare Source
Bug Fixes

sidebarsubmenu: issue w/ horiz padding on large screens (bf3b57d)

v3.43.2
Compare Source
Bug Fixes

modal: allows clicks to be called on portaled elements (7d1b9b2)
typeahead: ensures menu renders with correct width (b91eec9)

v3.43.1
Compare Source
Bug Fixes

export codesnippet from packages index (c35966c)

v3.43.0
Compare Source
Features

textinputwithbadges: adds component for rendering badges in a textinput (f54389b)

v3.42.0
Compare Source
Features

pageheader: allows pageheader to render children (459b989)

v3.41.0
Compare Source
Features

let fullscreen modal content fill remaining height (33fb2e6)

v3.40.0
Compare Source
Features

adds CodeSnippet and ClickToCopyButton components (c8be546)

Renovate configuration
:date: Schedule: At any time (no schedule defined).
:vertical_traffic_light: Automerge: Enabled.
:recycle: Rebasing: Whenever PR becomes conflicted, or if you modify the PR title to begin with "rebase!".
:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

Available now for Enterprise: Renovate Pro with real-time webhook handling and priority job queue.

:tada: This PR is included in version 2.153.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


support for ueberzugpp

ueberzugpp is a command line utility written in C++ which allows to draw images on terminals by using child windows or using sixel on supported terminals. This is a drop-in replacement for the now defunct ueberzug project.
I use it on my ranger, and it works out-of-box. But it do not directly work on rnvimr. Can rnvimr add support for ueberzugpp?

Look like doesn't work for me. I am ArchLinux user.

I posted an issue on ueberzugpp, its syntax is a bit different from    the original ueberzug. But I use it on plain ranger in cli, it just worked. I use it on Ubuntu 20.04 BTY.

After upgrading it, ranger can preview the image, but can't switch images after two previews. It doesn't seem stable enough for me. I leave this issue and will support it if the time is ripe.

OK, it can switch pictrues as many as possible on my computer. Anyway, just wait a bit, seems the maitainer is actively developing.

@kevinhwang91 does your terminal support sixel? then perhaps that's the issue. Current ranger doesn't support Ü++'s sixel implementation, I've opened a PR on ranger's side but no luck so far https://github.com/ranger/ranger/pull/2786.

@jstkdng Sorry, after rebooting my system, it works fine outside nvim terminal, but can't work with ranger inside nvim. Does ueberzugpp use a protocol to communicate with images?

U++ can receive commands through stdin (og ueberzug) or tcp (newer).
It's weird that ranger in neovim doesn't spawn U++. Is the config being modified?

You can reproduce the issue follow below steps:

Start ranger inside nvim:  nvim --clean +'term ranger --clean' +'startinsert';
Enable preview image in ranger: set preview_images_method ueberzug and set preview_images true;
Browse images;

Yes, I'm able to reproduce the issue.
It seems U++ can't correctly find the size in pixels of the neovim terminal. And it's crashing as soon as it is spawned

The issue should be fixed on v2.3.0

Confirmed. Rnvimr doesn't need to change anything. Please add an installation section for ueberzugpp, so Rnvimr can add docs for the new optional.

Hmm, right now the only way to install is to build it from source. There's the AUR package I'm maintaining but I have no package for other distributions.
Maybe a flatpak could be built for other distributions but I'm gonna have to do some research.


Popup not coming on successful submission

Challenge http://freecodecamp.com/challenges/waypoint-understand-boolean-values has an issue.
After I change the code and run it, the popup to take me to the next challenge is not coming even thought the tests pass and give correct result. I faced the same issue in other challenges. Sometimes in other challenges the popup might come but on clicking the button to the next challenge it does not move to the next challenge.
Hope you can fix this.Thank you.

Hey @syedsuhail is this still an issue for you? Even when you do a hard refresh or open it incognito mode?

@benschenker  After facing the popup problem I started using ctrl+enter to move to the next challenge . That worked all the time. I tried out a few challenges now and it seems to work.


Bug: Crash on initial app load after install

Issue Description
Seeing an app crash only on initial load from a fresh app install
There isn't much in the stack trace but we were able to track the root cause back to <GeoLocationContainer /> in app/index.js
Steps to Reproduce / Code Snippets

Remove app from sim/device
Install app on sim/device from branch v2.4-rc
This part varies slightly (possibly due to timers firing):
3a) 4/5 times the following error comes up right away, prior to the Home view loading
3b) 1/5 times Home view and modal views load as normal - the same error comes up after interacting with Allow Location access popup

Additional Information

Platform(s) (iOS, Android, or both?): both

Seems like the culprit was
// fire immediately on different thread
this.setTimeout(
this.getPermission(),
100
);

Moving this.getPermission() out of the timeout doesn't seem to cause any issues.
@jpknoll I think the idea for the timeout was for performance? But the app seems to be fine with taking it out, any thoughts/arguments on why we should try keeping the timeout?

closing this out as the issue is resolved, @jpknoll let us know if you have any last thoughts


Add support for partial updates and fix header bug

Hi, this is a pull request that fixes a bug with the Referer header being wrong (qBittorrent expects it to start with http://).
It also adds support for receiving partial updates from qBittorrent.
WebUI#poll will continuously poll using WebUI#sync while automatically incrementing the response id, yielding partial updates.

hey this is very good update thanks very much

@mkroman Thanks for the pull request! Sorry it's taken me so long to see it.
When I run the test suite against qBitTorrent 3.3.16 I get failures when it attempts to authenticate. Are you seeing this too?
When I run master against 3.3.16 it authenticates and the tests run successfully.

Sorry for the late reply.
I think when I wrote this patch I was using the master version of qBittorrent (prior to 4.x).
According to the WebUI API documentation, APIv14 added a hard restriction on the HTTP referrer.
Since the new stable is 4.0.3, is it still necessary to provide backwards compatibility?
I would also like to propose that instead of providing a host, port, user and pass, the user of the gem would instead specify a URI:
require 'uri'

client = QtbClient::WebUI.new("https://admin:adminadmin@qbittorrent-server.local:8080")

Or more verbose:
require 'uri'

url = URI 'https://qbittorrent-server.local:8080'
url.user = 'admin'
url.password = 'adminadmin'

client = QtbClient::WebUI.new(url)

The reason for this is that the referrer header in my patch always assumes a plain-text HTTP referrer, and I imagine this would fail if qbittorrent is behind a proxy with SSL/TLS configured, but I'm not sure.


Displaying Undefined on the chart

I installed your package and in the controller I used your graph example and the text "undefined" appears on the top left side. Can you please tell me how I can fix this?

in your controller use ->setSubtitle('Your Subtitle')

Thanks


CI/CD Deployment Center instructions do not match what I see - can't complete

Hello
re https://docs.microsoft.com/en-gb/azure/bot-service/bot-builder-deploy-az-cli?view=azure-bot-service-4.0&tabs=userassigned%2Cnewgroup%2Ccsharp
Having authorised Azure to GitHub I am attempting to Step 2. "After authorizing the AzureAppService, go back to Deployment Center in the Azure portal.
Select Continue."
The screen I see is the following, it does not match the screenshot showing steps 1 to 4.
I fill in the screen I see as I believe is correct (items are valid) but the "Settings" header shows an * (presumably something is missing?) and the 'Save' link does not respond. I'm unable to continue with the CI/CD set up so can't develop my bot.
I also don't see a match for these instructions ->
"On the Build provider page, select the build provider you want to use and select Continue.
On the Configure page, enter the required information and select Continue. The information required will depend on which source control service and build provider you chose.
On the Summary page, review the settings and then select Finish.
"
Please advise, thank you.

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1e9b10d5-ca65-d08c-2f11-ba62d66b8f80
Version Independent ID: 396d52c0-d864-afac-4d1c-fd6880a973a9
Content: Configure continuous deployment for Bot Service - Bot Service
Content Source: articles/bot-service-build-continuous-deployment.md
Service: bot-service
GitHub Login: @JonathanFingold
Microsoft Alias: iawilt

also worth mentioning: the instructions don't tell the user to create a GutHub repo for the bot prior to CI/CD setup attempt.

Update: I found some instructions here that are a better match: https://docs.microsoft.com/en-gb/azure/app-service/deploy-continuous-deployment?tabs=github
I did this change:

Not fully tested but it seemed to work.

Thanks for bringing this up. Investigating.

I've updated and tested the instructions. Changes should be live by end of day.

Thanks.


Active development status

@justinbleach I understand that you must be occupied with other activities and have a busy schedule. I just wanted to check if we are planning any active development, as there are many great PRs raised to address the issues and enhance the library.

Hey @arpitshah2506  sorry yeah have been a little busy. I will take a look at a few PRs and merge in. Let me know if there is on in particular you are wanting in

@justinbleach can you please look into this as ADFS vendor has logout binding as redirect, so when data sent using browserutils it is throwing csp error, I found PR related to it :
https://github.com/justinbleach/saml-client/pull/59/
Please take a look at it


Reading 'foo'.length fails

Code that reads from a string literal's length property triggers errors from this plugin.

$ npm install -g eslint eslint-plugin-no-use-extend-native
$ eslint --version
3.2.2

This code should pass, but does not, regardless of wrapping it in parentheses, etc.
$ echo "console.log('foo'.length)" | eslint --stdin --rule='no-use-extend-native/no-use-extend-native:2' --plugin='no-use-extend-native'

<text>
  1:13  error  Avoid using extended native objects  no-use-extend-native/no-use-extend-native

✖ 1 problem (1 error, 0 warnings)

This code should also pass and does, successfully.
$ echo "var x = 'foo'; console.log(x.length)" | eslint --stdin --rule='no-use-extend-native/no-use-extend-native:2' --plugin='no-use-extend-native'
$

Thanks for reporting this.
Interesting that
var x = 'foo'.length;

works, but
console.log('foo'.length)

does not.
I'll try to take a look at this tonight after work.

Ok, looked at it this morning. Should be released soon. Just waiting for Travis to run tests.


Update "How to Use" video

In the "How to Use" video, the route "/addpeer" is demonstrated instead of "/addPeers".
Also the example body contents should be {"peers":["address1", "address2",...]} or {"peers":["address1"]} not {"peer":"address1"} if the correct route "/addPeers" will be shown.
Thanks!

Thx! The current video is based on onechain V1... but now V2.
Please refer to the text mainly before updating.
👍


🔶 Add Sublime Text

Closes #13

By the way, there is also Vimium extension for browsers. It has vomnibar feature which is used to open URL, bookmark or history entry.
I could make a PR if you consider it appropriate 😃

Re: vimium. Sure why not. :)


Question:

Problem
I can't see all the keybinds in the controls menu it only shows "open calculator"

Need more info: what version of the mod? what version of minecraft? maybe some screenshots
Here is the latest version of the mod and everything is working properly:

latest verison 1.20.4

in the version for 1.20.4 there is only a calculator)
I was just learning how to create mods back then and that's all I managed to do on this version. everything will work on newer versions of minecraft. if necessary, I can do a backport for this version, but it won't be soon, because lately I don't have much time at all

if you have time backporting it would be great but no rush

no i need i can update myself


Download latest PowerShell Core build for Travis-CI builds

Hi @bmanikm, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;


[TOPI] Add tensor multiplication.

Thanks for contributing to TVM!   Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers.

Thanks for the contribution! Please request reviews from reviewers

@nishi-t @were Could you please help in reviewing.

@nishi-t thanks for review, I've updated patch. Now it fully compatible with numpy's tensordot. Could you help in reviewing again?

@Vooblin Sorry for the late reply. I'll review this in a few days.

Thanks, @Vooblin @nishi-t , this is merged!


Fix score

Fixes #4
This fixes the score by adding the score text inside a box with the same color of the background.
The box is then drawn every time at the same position, creationg the illusion that the text is changing.

@adalbertogonzaga , should I open this to main or dev?

Looks good to me 👍🏼


Create vsix include .dll

I try to include some .dll file to vxis, but it is fail.
1>EXEC : error : spawn REG ENOENT
1>      at exports._errnoException (util.js:870:11)
1>      at Process.ChildProcess._handle.onexit (internal/child_process.js:178:32)
1>      at onErrorNT (internal/child_process.js:344:16)
1>      at nextTickCallbackWith2Args (node.js:442:9)
1>      at process._tickCallback (node.js:356:17)
I added the following line to VsixManifestBuilder.CONTENT_TYPE_MAP
".dll": "application/octet-stream"
It is working fine.

Fix available in next release.


fix preloadReady

How is this fixing it?

Without this fix I have this error:

And with this fix I have all the files that load, not just those concerned.

Can you post exactly where in the react-loadable code that error is thrown? I can't tell from the stack trace because line numbers are based on your bundle.

I do not have the error above, but on the other hand without this fix I have the following error:
Warning: Text content did not match. Server: "ChatSend" Client: "Loading"

Otherwise the rest works perfectly well, the reason why all my routes loaded is that all my Loadable were in the same file...

the reason why all my routes loaded is that all my Loadable were in the same file...

It shouldn't matter if they are all declared in the same file, it should only matter if they get rendered

Now I do not have the same client-side rendering as server side, I have the loading components at first render in the client without this fix.

I'm not sure why because this change does the exact opposite of what should be done. The updated test case is now incorrect behavior. Without an example of what you're doing I'm going to assume it is something wrong at your end and this just happens to fix it even if it introduces bad behavior.