Update Pylance settings telemetry property list to match code

I needed to add the disableTaggedHints property to the settings event and noticed that several other properties were also missing. Added them. Also fixed alphabetical sorting.

@debonte Should this be merged for this iteration?

Erik's OOF today. I'd say yes. We already shipped this telemetry.


Mobile/Desktop Safari recorded audio doesn't playback in Mobile Safari

Confirmed working:

Record: Desktop Safari / Playback: Desktop Safari
Record: Mobile Safari / Playback: Desktop Safari

Broken:

Record: Mobile Safari / Playback: Mobile Safari
Record: Desktop Safari / Playback: Mobile Safari

Behavior: Recording "waveform" and play button is shown, but nothing happens when touched. Can't move the cursor when attempting to seek.
If you'd like to, I'm happy to sit down and do a live debug session. If you have access to a Mac, will probably also repro in the iPhone simulator.

Strangely, upon recording in Mobile Safari (before pressing send) the waveform doesn't appear, and the playback button is replaced with the "no playback" symbol

I see cryptic errors in the console, without traces.
[Error] Unhandled Promise Rejection: EncodingError: Decoding failed

	promiseEmptyOnRejected
	promiseReactionJob

There is no simple way to debug it. Given that Apple intentionally cripples iOS Safari (for example, no push notifications) I'm inclined to just let it slide unless more people request support for it or someone pays.


Bug: UInt32 causes JSONModel to throw exception on some devices

I am using a UInt32 property in my JSONModel class to handle integer data that needs to be 32 bits in length. I am also using propertyIsIgnored: to ignore this property.
These are the devices I am using JSONModel 1.0.2 on these devices

iPad 3rd Generation (32 bit) iOS 8.3
iPhone 5s (64 bit) iOS 8.3

This will work on my iPhone but when I run it on the iPad I get the following error output:

*** Terminating app due to uncaught exception 'JSONModelProperty type not allowed', reason: 'Property type of <class>.<UInt32 property> is not supported by JSONModel.'

I did some debugging and I tracked it down to __inspectProperties in JSONModel.m right before the exception is thrown. The iPad's UInt32 propertyType is L which is not in valueTransformer.primitivesNames, whereas the iPhone's UInt32 propertyType is I which is and is an NSInteger.
Adding @"L": @"long" to _primitaveNames in JSONValueTransformer.m seems to solve the issue.
_primitivesNames = @{@"f":@"float", @"i":@"int", @"d":@"double", @"l":@"long", @"c":@"BOOL", @"s":@"short", @"q":@"long",
                             //and some famos aliases of primitive types
                             // BOOL is now "B" on iOS __LP64 builds
                             @"I":@"NSInteger", @"Q":@"NSUInteger", @"B":@"BOOL",

                             // 32 bit int in 32 bit iOS
                             @"L": @"long",

                             @"@?":@"Block"};

Having thought about this quite a few times over the last year or so, I have come to the conclusion that JSONModel doesn't (and shouldn't) support particularly specific property types. It is intended to deserialize into an instance of a class which directly corresponds with the JSON data.
As such, only NSInteger & NSUInteger are supported at present.


The admin portal is not opening. showing as access denied

Admin Portal is access denied
I clone the source code and launched in visual studio. The admin portal portal page is showing as access denied..

Hi @NareshkumarBT , please add yourself as KnownUsers in the app service config or add your email to the database.

Thankyou. I added my user in Knownusers and could able to login?

Hi @NareshkumarBT , I am also facing the same issue even though I have added my email ID in the config as KnownUsers

Please let me know how you were able to resolve this issue.


e-mail Configuration

Hi
please anyone help me to configure email with this image.
When i m trying to add new user, it is not sending any request to newly created user.

This thing is still a problem that in fact prevents to use this image productively.

Then please suggest any alternate way to do the same.....
or any better working image with steps....
Thanks :)

docker run -d --name taiga-back  -v "$(pwd)"/backend/docker-settings.py:/taiga/settings/local.py:ro -p 8000:8000 --link postgres:postgres ipedrazas/taiga-back

then you can change the configuration in the file backend/docker-settings.py
# MAIL OPTIONS
DEFAULT_FROM_EMAIL =<EMAIL_ADDRESS>#By default is using the console
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"

EMAIL_USE_TLS = True
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_PORT = 587
EMAIL_HOST_USER =<EMAIL_ADDRESS>EMAIL_HOST_PASSWORD = 'supersecret'

DJMAIL_REAL_BACKEND = "django.core.mail.backends.console.EmailBackend"

closing


Groups related configuration

Is your feature request related to a problem?
Yes, If I want to have custom gruops or group order I can't (and this is a base TypeDoc feature)
Describe the solution you'd like
Make this possible
Describe alternatives you've considered
No response
Additional Context

Also doesn't work excludeGroups, I think would be great also has custom groups defined by the user using @group in jsdoc
Versions:
"starlight-typedoc": "^0.11.0",
"typedoc": "^0.25.12",
"typedoc-plugin-markdown": "4.0.0-next.54",

Thanks for your report.
Additional TypeDoc configuration can be passed down to the plugin using the typeDoc configuration.
For example, running pnpm dev in the examples/ folder of this repository will generate the following sidebar by default:

Adding the groupOrder configuration you mentioned can be done in your Astro configuration file:
starlightTypeDoc({
  // Other configuration…
  typeDoc: {
+    groupOrder: ['Functions', 'Classes', 'Type Aliases', 'Interfaces'],
  },
}),

After these changes, the generated sidebar will look like this:

The order of the groups matches the one provided and extra groups are appended at the end as documented in the TypeDoc documentation.

And custom group names are avaible

https://typedoc.org/options/organization/#categorizebygroup


[5.4] withoutGlobalScopes is not works in eager loading if morphTo relation is used

Laravel Version: 5.4.12
PHP Version: 7.0.14
Database Driver & Version: pdo_pgsql 7.0.14

Description:
There is no way to omit global scopes for morphTo relation.
Also the same problem is described here: https://laracasts.com/index.php/discuss/channels/general-discussion/remove-global-scope-on-morphto
Steps To Reproduce:
Schema::create('my_models', function (Blueprint $table) {
    $table->increments('id');
    $table->integer('parent_id');
    $table->morphs('owner');
});

...

class MyModel extends Model {
    public function parent() { // Simple belongsTo relation
        $relation = $this->belongsTo(Parent::class);
        $relation->withoutGlobalScopes(); // Works fine
        return $relation;
    }
    public function owner() { // Simple morphTo relation
        $relation = $this->morphTo();
        $relation->withoutGlobalScopes(); // No effect
        return $relation;
    }
}

...

$model = MyModel::with('parent', 'owner')->first();
assert(!is_null($model->parent); // ok
assert(!is_null($model->owner); // fails if owner was filtered by any global scope

+1

+1

+1


Consensus on naming and behaviour of sections and subsections in the Collective page

During the design and implementation of the new navigation for Collectives pages we have been encountering some discrepancies in how we name and treat the sections of the Collective page.
We need to solve it in a language level before moving forward with the implementation of the project.
Currently we have this organization (Example):
We see these sections in the navigation

Whereas we have the following (also) sections in the settings page where you manage the order of sections of your page:

One of the main goals of the new design proposal is to simplify sections in the high level, so we narrowed down to just 5 main sections:

Plus, we are adding the rest of the features of the Collective page as 'subsections'
@Betree suggested to name the main sections rather as 'Categories' and the subsections as 'features', we want to raise the discussion first to decide how are we naming these different nesting levels before moving forward with the implementation.
Here is a table that displays all the categories and features, and which ones have stand alone pages (even if they are not included in production today).

Profile page sections
Standalone page

Contribute/Contributions

Make financial contributions (Tier cards on COLLECTIVE/ORG, TIckets on EVENT)
✅

Contributors ('All of us')
🚫

Financial contributions made
🚫

Recurring contributions made
✅

Events/Projects

Events
🚫 (doesn't have one yet, but should it?)

Projects
🚫 (only on type FUND)

Budget/Transactions (name of category for type USER)

Transparency
✅

Transactions
✅

Expenses
✅

Goals
🚫

Top Financial Contributors
🚫

Connect

Updates
✅

Conversations
✅

About

Team (Core contributors)
🚫

Connected collectives
🚫

About
🚫

(* Edited by @sbinlondon to add alternatives to main 5 categories based on collective type and some missed out sections/features)
As we see here, we have three different items:

Example
Suggested name
Behavior

Top level item
'Category'
Takes you to the Collective page and scrolls you down to that category

Sub-index w/page ✅
'Menu item'
Takes you to the standalone page of the feature

Sub-index w/no page 🚫
'Feature'
N/A It just lives in the C.Page

With this proposal, we will have the following dropdown menus for desktop, and an expanded menu for mobile (Not collapsable)

Desktop
Mobile

@znarf @Betree @kewitz @sbinlondon  @alanna @piamancini

Re: @Memo-Es's suggestion:

Example
Suggested name
Behavior

Top level item
'Category'
Takes you to the Collective page and scrolls you down to that category

Sub-index w/page ✅
'Menu item'
Takes you to the standalone page of the feature

Sub-index w/no page 🚫
'Feature'
N/A It just lives in the C.Page

Largely agree on the divisions, disagree on the names and what they imply.
The terminology issue may have been complicated by my decision to use @Betree's features endpoint work in this project but it's a decision I stand by and think will make things more simple in the end.
The features endpoint is to help us keep the logic about what a collective can do (i.e. receive financial contributions as a COLLECTIVE or ORG, receive expenses as a host ORG); and what data is displayed (i.e. collective type can have Conversations, Updates, and Goals but has a few Conversations threads, has Goals disabled, and no published Updates despite the feature being enabled, therefore we would only display Conversations on their page). Features is mostly an API thing.
I think a good framework would be:

Feature: used to talk about what a collective is allowed to do (receive contributions or expenses, make contributions, have goals, have conversations), whether it has its allowed features enabled or disabled, and if enabled whether they have data (are "active")

Category: the main clickable words that are listed on the navbar and as headings with cute icons on the collective page

Section: discrete blocks within the broader categories on the collective page, which we may choose to show/render based on the collective type and its available features and data (this might sometimes map directly to a feature like Conversations or it might be a more widgety thing like Top Financial Contributors).

Expandable section: if a section on the collective page has an equivalent standalone page (i.e. the "Transactions" section in the "Budget" category has "View all transactions" which links to /collective/transactions) (I'm not super attached to the name "expandable section", think it needs work, but I do agree with Memo's idea that we should have a separate name for the sections that are also standalone pages)

Using this, I think the main source of confusion for some people (and potential confusion with the users) was that the categories on the navbar and mobile menu would act differently depending on which page we were on:

on the collective page if you click a category like "Connect" it scrolls you down to that category heading on the page. No drop-down menu appears.
if you are on a standalone page like /collective/transactions and click the category "Connect" then the dropdown menu  would appear and you can click to go to the standalone pages for "Conversations" or "Updates"

It seems we're proposing to solve this by the drop-down menus always appearing on desktop/tablet, and the entire menu being visible on mobile? Or do we want to think more about how we can clearly signal between being able to click a category to see its expandable sections vs. clicking a category to go to that category heading on the collective page?
Another outstanding question is how much control do we give users over their profile page? Can they reorder/hide entire categories on the page, but not the sections within them? Can they reorder/hide both categories and sections within categories?

@piamancini @Betree @kewitz @znarf

This brings more clarity, thank you for documenting that @Memo-Es and @sbinlondon.
I agree with everything described in https://github.com/opencollective/opencollective/issues/3698#issuecomment-731140874. My main feedback would be to be careful with using just Category, because this word already has a meaning in the context of accounts (categories like casino, seo, etc). Out of clarity, I think we should refer to them as "Section category".

It seems we're proposing to solve this by the drop-down menus always appearing on desktop/tablet, and the entire menu being visible on mobile?

I think that's the way to go for first iteration.

Another outstanding question is how much control do we give users over their profile page? Can they reorder/hide entire categories on the page, but not the sections within them? Can they reorder/hide both categories and sections within categories?

That will be the most complex part, especially because we'll need to migrate the existing sections to a new format like:
sections = [
  ABOUT: [
    { section: ABOUT, isEnabled: true },
  ],
  BUDGET: [
    { section: TRANSACTIONS, isEnabled: true },
    { section: EXPENSES, isEnabled: true },
  ]
]

To me we should be able to re-order section categories (ABOUT, BUDGET, etc) and sections inside of them. But you can't get a section out of it's Section category.
Translated to UI:

@Memo-Es the expanded mobile menu is looking really good, I think it solves the issue related to linking to pages and sections we discussed in our meeting.
I agree with @sbinlondon. It is important to set coherent, non-conflicting names and be consistent across the front and back end.
My problem with listing standalone pages as a link in the dropdown is that sections belong to pages and not the other way around.
This doesn't look very important because it still represents a 1:1 relationship, but this will cause you trouble when navigating across standalone pages because all these sections will make no sense on a different page.
I can see two ways of solving this:

This would require a menu that mutates across pages.

Keep the same navigation bar (and possibly the collective header) across all standalone pages to give the feeling these pages are full-screen components of the same collective page.

@Betree I think that's an elegant data-structure to address this issue.

Another outstanding question is how much control do we give users over their profile page? Can they reorder/hide entire categories on the page, but not the sections within them? Can they reorder/hide both categories and sections within categories?

That will be the most complex part, especially because we'll need to migrate the existing sections to a new format like:
sections = [
  ABOUT: [
    { section: ABOUT, isEnabled: true },
  ],
  BUDGET: [
    { section: TRANSACTIONS, isEnabled: true },
    { section: EXPENSES, isEnabled: true },
  ]
]

This is fine by me. I want extra extra clarity though:

This sections array is the one in settings.collectivePage right?
Does the isEnabled on a section in the settings.collectivePage.sections array ONLY pertain to whether that section is visible/not visible on the collective page?
settings.collectivePage.sections[section].isEnabled is different to settings.features[feature].enabled? What happens if we have settings { features { conversations: false } } because the collective has opted out of having conversations but

settings {
  collectivePage: {
    sectionCategories: {[
      CONNECT: [
        { section: CONVERSATIONS, isEnabled: true },
      ]
    ]}
  }
}

wouldn't have an effect because we're opting out of the feature? Or would we have logic on the API side to say when we opt out of conversations via settings.features.conversations it also sets the bit in settings.collectivePage.sectionCategories... to false? Or maybe vice versa where if they hide conversations from their collective page in this proposed menu below, that is effectively seen as opting out of the feature?

To me we should be able to re-order section categories (ABOUT, BUDGET, etc) and sections inside of them. But you can't get a section out of it's Section category.
Translated to UI:

That's pretty much what I was imagining for the edit profile page menu, thank you for visualising! 🙌
I agree with @kewitz that navigation should be consistent wherever it's used. We don't mutate the top bar profile menu's design or functionality across pages, and I feel we should try our best to keep a similar consistency with the collective navigation.

Agree about the navigation consistency. The proposal described above is meant to be consistent on all Collective's instances, from the Collective page and all of its possible sub-pages.
So if you click on any of the 'Section Categories' like 'Budget,' it will take you to the Collective page, down to the 'Budget' section on both desktop/mobile, and only 'Expandable sections' using @sbinlondon term will be displayed in a dropdown.
One thing also I want to surface is @alanna 's comment about the 'Goals' feature:

Don’t combine goals and budget - some people need to put the goals progress bar right at the top and want the budget stuff elsewhere.
Basically, I think we should split up things to customize, reorder, and hide/show what they want more flexibly.

I don't think we should make 'Goals' a section of its own. These types of small decisions generated a lot of complexity in the current Collective Navigation, which was one of the main points of the design need for this update. Anyway, I wonder if that could be a widget integrated to the top part of the Collective page. @Betree, any thoughts on this?
(Also, we should also remember that we brought back this feature after a couple of negative reactions from users, so if we want to keep it, we should rethink how it plays out with the new stuff that we are planning for the near future. I decided to put it under budget because it will also be featured in the new 'Transparency' page for Collectives for sure.

@sbinlondon

This sections array is the one in settings.collectivePage right?

Yes

Does the isEnabled on a section in the settings.collectivePage.sections array ONLY pertain to whether that section is visible/not visible on the collective page?

Currently yes, but we may want to keep https://github.com/opencollective/opencollective/issues/3396 in mind - though I think https://github.com/opencollective/opencollective/issues/3396 will be invalidated by the fact that we'll soon have CONNECT instead of UPDATES/CONVERSATIONS.

settings.collectivePage.sections[section].isEnabled is different to settings.features[feature].enabled?

It currently is:

collectivePage.sections[section].isEnabled > Section (ie. Buget, Contributors, About, etc)
settings.features[feature].enabled > Feature (ie. ReceiveExpenses, ReceiveContributions, etc)

What happens if we have settings { features { conversations: false } } because the collective has opted out of having conversations but

It's the issue we already discussed in https://github.com/opencollective/opencollective/issues/3396. There are multiple ways to solve that, we could for example delegate the value for isEnabled to the settings value:
settings: {
  collectivePage: {
    sections: [
      { type: 'SECTION', section: 'GOALS' },
      {
        type: 'SECTION_CATEGORY',
        category: 'CONNECTED',
        sections: [
          { section: CONVERSATIONS, isEnabledSettingKey: 'features.conversations.enabled' },
        ]
      }
  ]
}

But if it introduces too much complexity we can solve that in a separate PR.

Or maybe vice versa where if they hide conversations from their collective page in this proposed menu below, that is effectively seen as opting out of the feature?

That's another option

@Memo-Es

Anyway, I wonder if that could be a widget integrated to the top part of the Collective page. @Betree, any thoughts on this?

If I understand correctly, what you describe as a "widget" is a sections that wouldn't be integrated in the navigation right? If that's correct, it's indeed possible. These Widgets could be sections that can live outside of a Section category. So:
settings: {
  collectivePage: {
    sections: [
      { type: 'SECTION', section: 'GOALS' },
      {
        type: 'SECTION_CATEGORY',
        category: 'CONNECTED',
        sections: [
          { section: CONVERSATIONS, isEnabled: true },
          { section: UPDADTES, isEnabled: true }
        ]
      }
  ]
}

@sbinlondon

This sections array is the one in settings.collectivePage right?

Yes

Does the isEnabled on a section in the settings.collectivePage.sections array ONLY pertain to whether that section is visible/not visible on the collective page?

Currently yes, but we may want to keep #3396 in mind - though I think #3396 will be invalidated by the fact that we'll soon have CONNECT instead of UPDATES/CONVERSATIONS.

What happens if we have settings { features { conversations: false } } because the collective has opted out of having conversations but

It's the issue we already discussed in #3396. There are multiple ways to solve that, we could for example delegate the value for isEnabled to the settings value:
settings: {
  collectivePage: {
    sections: [
      { type: 'SECTION', section: 'GOALS' },
      {
        type: 'SECTION_CATEGORY',
        category: 'CONNECTED',
        sections: [
          { section: CONVERSATIONS, isEnabledSettingKey: 'features.conversations.enabled' },
        ]
      }
  ]
}

But if it introduces too much complexity we can solve that in a separate PR.

No, I really like this idea. I think what I'm looking for is one source of truth, i.e. one place we can look to find all this info, and this looks like the ideal solution because 1) it makes use of the different keys we have now and means we wouldn't have as many problems trying to migrate everything to one or the other and 2) does collate everything regarding the collective page into one place.

@Betree Since I am working on the edit collective page settings thing mentioned above as a significant portion of this project, would I be able to have you work on these: https://github.com/opencollective/opencollective/issues/3641

Sure, but we should probably solve the issue about how we're going to store/process the settings to render these first. The way will solve it will impact the way we develop these menus.

Sure, I think we've all approached some conclusions but happy to summarize and try to move forward:

The navigation structure

We have section categories (i.e. Budget, Contribute, About) which are the main elements on the navbar and the heading on the collective page.

We have sections within these section categories (i.e. sections Transactions, Expenses within section category Budget)

For Goals and potentially Top Contributors, we have a separate type called a widget which 1) does not appear in the navbar or dropdown menu and 2) can be moved anywhere on the collective page and 3) shown/hidden on its own and not having to consider what section category it is in (so Goals won't be in Budget anymore)

The navbar visuals/layout/behaviour

We have agreed that the navigation should act the same way on all pages
We have decided as an MVP that there is the non-collapsible mobile menu, and for desktop/tablet the drop-down menus
If we click a section category like Budget, it opens the drop-down menus so you can select Transctions or Expenses and go to those pages
If you are already on the collective page, clicking Budget won't scroll you down to that section (so we have the behaviour matching on all pages)

The edit collective settings

We are going to store/represent the structure of the collective page in the database like so:

settings: {
  collectivePage: {
    sections: [
      { type: 'SECTION', section: 'GOALS' }, // a.k.a. a 'widget'
      {
        type: 'SECTION_CATEGORY',
        category: 'CONNECT',
        sections: [
          { section: CONVERSATIONS, isEnabledSettingKey: 'features.conversations.enabled' },
          { section: UPDATES, isEnabledSettingKey: 'features.updates.enabled' },
        ]
      }
  ]
}

Visually on the edit collective profile page, it will look like this, allowing users to reorder section categories and widgets on the page, and reorder sections within the section categories. They can hide (enable/disable) widgets and sections but not sections categories.

All good, feel free to assign me on the API issues.
I'll probably start by implementing a convertSectionsToNewFormat function in the frontend so that we can work with the new settings.collectivePage.sections without having to wait for API to be fully implemented.

Thank you @sbinlondon for the summary! Just a small detail:

If we click a section category like Budget, it opens the drop-down menus so you can select Transactions or Expenses and go to those pages
If you are already on the collective page, clicking Budget won't scroll you down to that section (so we have the behavior matching on all pages)

The dropdowns will appear while hovering on each navigation item (Unless there are no dropdown elements to show). If you click on the main item it will scroll you down to that section if you are on the Collective page or take you to the collective page, down to that section if you're in a sub-page
cc @Betree

Clarifying after stand-up (thanks @Memo-Es and @piamancini!):

Clicking a category section in the navbar always takes you to that section on the collective page
Hovering on a category section in the navbar opens the section menu
The mobile menu is uncollapsible for the above reasons, since you can't hover on mobile. Behaviours are the same.


ngAtomicNotify directive DI problem

When I try to compress the lib using uglifier with mangle: true I get the error "Unknown Provider", because during the compression "$scope" parameter name has been changing here https://github.com/maxigimenez/angular-atomic-notify/blob/master/src/angular-atomic-notify.js#L102
The possible solution is to use DI in this line https://github.com/maxigimenez/angular-atomic-notify/blob/master/src/angular-atomic-notify.js#L128
Like that ['$scope', controller]
I also can reproduce the problem even without a compression just using ng-strict-di directive. Then it says "Error: [$injector:strictdi] function($scope) is not using explicit annotation and cannot be invoked in strict mode"
Should I create a PR or you can fix it by yourself?

Fixed on #12


[Feature Request] Don't include the number of files in the search count

Thank you for developing this wonderful extension! I have been using it every day instead of telescope's live_grep since I came across this extension.
Is your feature request related to a problem? Please describe.
Thank you for developing this wonderful extension. I have been using it every day instead of telescope's live_grep since I came across this extention.
Describe the solution you'd like
I would like to suggest that the number of search results should only show the number of matches (as it currently shows the number of matches + number of files).
Or is it possible to display somewhere the number of files matched or the number of matches? (like Results (4) for example)
I am not familiar with telescope's extensions, so maybe this is an impossible request. Also, if this is the intended specification, please feel free to close this issue.
Describe alternatives you've considered
Additional context

Thank you for developing this wonderful extension! I have been using it every day instead of telescope's live_grep since I came across this extension.

Thank you for the kind words :) glad you enjoy the extension!
Your feature request is very sensible, I will hopefully have some time to look into it next week.

The issue should now be fixed, I accidentally pushed it right to master rather than a feature branch :sweat_smile: but it should work all the same :)

@fdschmidt93
Thanks so much for the quick fix.

I accidentally pushed it right to master rather than a feature branch 😅

I make the same mistake sometimes when I'm doing personal development projects too. haha
I've actually tried it and I'm so glad it's more convenient!
Thank you so much!


SABPostProcess python path is incorrect in OSX

Issue
I was having some difficulty with SABNZBD returning errors stating it was missing modules.

Traceback (most recent call last):
File "/Volumes/MediaBay/mp4_automator/SABPostProcess.py", line 5, in
from autoprocess import autoProcessTV, autoProcessMovie, autoProcessTVSR, sonarr, radarr
File "/Volumes/MediaBay/mp4_automator/autoprocess/autoProcessMovie.py", line 11, in
import requests
ImportError: No module named requests

Troubleshooting
Verified that all modules were installed and properly working by confirming no issues when using manual.py
I think the root cause is due to sabnznbd's python script using /usr/bin/python as seen here, and the automators sabpostprocess script making use of env to grab the path's.
The issue is more than likely isolated around OSX as the version of python included in OSX has issues with some of the modules (requests[security]) and most users use an alternate version via homebrew which places its path elsewhere /usr/local/bin/python.
Workaround
I'm not sure if this is something that can be accounted for possibly by adding an override in the ini for the python path, or possibly something similar to the sickrage workaround including the path in the configuration page.
Currently I have modified the first line of the SABPostProcess.py path to point directly to the homebrew build of python which has all the required modules, but this path would be different for different people and not the best solution.
Verified working after changing path:

2017-07-14 09:46:30 - SABPostProcess - INFO - SAB post processing started.
2017-07-14 09:46:30 - readSettings - INFO - /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
2017-07-14 09:46:30 - readSettings - WARNING - Audio bitrate was invalid, defaulting to 256 per channel.
2017-07-14 09:46:30 - SABPostProcess - INFO - Performing conversion
2017-07-14 09:46:30 - SABPostProcess - INFO - Processing file /Volumes/MediaBay/Downloads/complete/TV/Arrow.S05E12.1080p.WEB-DL.DD5.1.H.264-DRACULA.4/170118_14.mkv.
2017-07-14 09:46:30 - mkvtomp4 - INFO - Reading video stream.
.....
2017-07-14 09:46:31 - mkvtomp4 - INFO - Reading audio streams.
......
2017-07-14 09:46:31 - mkvtomp4 - INFO - Starting conversion.
2017-07-14 09:48:27 - mkvtomp4 - INFO - /Volumes/MediaBay/Downloads/complete/TV/Arrow.S05E12.1080p.WEB-DL.DD5.1.H.264-DRACULA.4/170118_14.mp4 created.
2017-07-14 09:48:27 - SABPostProcess - INFO - Passing /Volumes/MediaBay/Downloads/complete/TV/Arrow.S05E12.1080p.WEB-DL.DD5.1.H.264-DRACULA.4 directory to Sonarr.
2017-07-14 09:48:27 - autoprocess.sonarr - INFO - Sonarr notifier started.
2017-07-14 09:48:27 - autoprocess.sonarr - INFO - Requesting Sonarr to scan directory '/Volumes/MediaBay/Downloads/complete/TV/Arrow.S05E12.1080p.WEB-DL.DD5.1.H.264-DRACULA.4'.
2017-07-14 09:48:27 - autoprocess.sonarr - INFO - Sonarr response: queued.

Unfortunately there's no good workaround to this, you either have to modify the shebang (the first line of the file) or use virtual environments. This is just how OSX handles its python installation
This thread had a good discussion
https://stackoverflow.com/questions/16503761/python-shebang-problems-with-importing-modules-and-choosing-the-correct-interpr

That was my assumption also, the other alternative is to not use the pre-built SABNZBD app and to run it the same way you would within a linux system using the source files.


NuttX CI build testing

Tests lists and makefile: https://bitbucket.org/david_s5/nuttx-knot/src (move to https://github.com/PX4-NuttX/nuttx-build-testing)
Setup on semaphoreci or travis-ci:
https://semaphoreci.com/nuttx/nuttx-knot/branches/master/builds/31

Done, see Jenkins commit carried in the PX4 branches. Would be great to get this supported in upstream NuttX eventually.


Update Docker Command

The Docker Options should be seperate from the arguments provided to the
container, and need to be specified separately so the image can be
placed in the appropriate place.

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.

R: @bsidhom @herohde (either)


fix: update enrolment confirmation messages

PT-1782.
Remove the part of the message that mentions that the site is stil ludner development.

kultus-ui branch is deployed to platta: https://kultus-ui-pr329.dev.hel.ninja :rocket::rocket::rocket:


Show/Hide Interaction Bar on MouseOver

With many people using Nice as a desktop app, it seems only natural to make use of the mouseOver/mouseOut functions. Some people have asked that the interaction bar not need a click before being visible, and one of the simplest ways to do this would be to show the interaction bar only when the mouse is over an item. The interaction bar should disappear within X seconds afterwards, though, to ensure the screen is not full of interaction bars.

This functionality has been added. While testing, it was found the function could be a bit of a nuisance. For this reason it is Disabled by default. Should a person want to enable it, the function is in the Preferences Panel under "Hover".


check its working

Silly question - but everything seems correctly installed, the log file says
Healthchecking Cron Service
every couple of minutes
the folders are create - but the schedule folder is empty
in admin the table is empty
the cron table is empty
So how can i check its working as expected? Any ideas?

Manually running php bin/magento cron:run seems to have got it going

although the crons are there working
still concerned something isn't quite right and i am missing something

@mattyl Did you create the Magento crontab as per https://devdocs.magento.com/guides/v2.2/config-guide/cli/config-cli-subcommands-cron.html?

yes! I installed this because of the problem with crons getting stuck and crashing the server - but the crons were running fine
in case i am being too clever and missing sometihng obvious

/opt/cpanel/ea-php71/root/usr/bin/php /home/keene/public_html/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /home/keene/public_html/var/log/magento.cron.log

/opt/cpanel/ea-php71/root/usr/bin/php /home/keene/public_html/update/cron.php >> /home/keene/public_html/var/log/update.cron.log

/opt/cpanel/ea-php71/root/usr/bin/php /home/keene/public_html/bin/magento setup:cron:run >> /home/keene/public_html/var/log/setup.cron.log

is my crontab

I also changed php to /opt/cpanel/ea-php71/root/usr/bin/php in the settings in case that was the issue

is this process meant to run and run? when i did it manually everything worked, but my CSF firewall was moaning about a never ending process - when i killed that process it never started again

Yes, it is running as a service so it will run until killed or the heat death of the universe.

Ah - but is there way of initiating it without running manually php bin/magento cron:run
i think this would be useful in the readme notes, i am running a standard cpanel server with the CSF firewall - it sends me messages when a process lasts more than x seconds
unless i manually ignore it in the pignore.txt
which i will do now!
But you can see how worrisome that would be if you didn’t know.

On 28 Jun 2018, at 17:58, Martin Pachol<EMAIL_ADDRESS>wrote:
Yes, it is running as a service so it will run until killed or the heat death of the universe.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magemojo/m2-ce-cron/issues/41#issuecomment-401084749, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbLAzUZ4J__xQ2EaKb3nabDE3TVI5PCks5uBP0JgaJpZM4U7NzT.

Just running in a cron the way you have should be working. Might need to check your /var/cron logs to see if there is some other error.

Thanks for that, it seems to be running ok, i removed my old crons and used the magento script to reinstall - they were identical but now it has the
#~ MAGENTO START
#~ MAGENTO END
comments in the crontab
could this be the issue? For me it seems ot be working and I have whitelisted the 3 processes - but for those following - if the comments are necessary for some reason it would be good to know! thanks for your help

I can't think how that would make any difference to it. Comments in the crontab wouldn't effect the code running from it. Maybe some cpanel peculiarity.

I am really sorry there is an issue somewhere here. The service stopped (due to an error) over night - it doesnt restart
the cron returns
Healthchecking Cron Service
But it only a manual intervention does it also return
Starting Service
I appreciate there must be something in my setup that is stopping the service starting, any idea how i can test this?

On 28 Jun 2018, at 22:50, Martin Pachol<EMAIL_ADDRESS>wrote:
I can't think how that would make any difference to it. Comments in the crontab wouldn't effect the code running from it. Maybe some cpanel peculiarity.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magemojo/m2-ce-cron/issues/41#issuecomment-401169113, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbLA_XtU_jVsuCB3fuLATgK_vdss0q_ks5uBUGagaJpZM4U7NzT.

Is there anything in the system cron log that is saying why it isn't starting?

No, sorry, that’s why I was raising it as an issue, have checked all logs
The cron is running and triggering the health check notice, but it’s not starting the service .
Sent from my iPad

On 29 Jun 2018, at 22:04, Martin Pachol<EMAIL_ADDRESS>wrote:
Is there anything in the system cron log that is saying why it isn't starting?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Hi, absolutely nothing
but its failing every day at 1.00am
[Exception]
Notice: Undefined index: method in /home/keene/public_html/vendor/magemojo/m2-ce-cron/Model/Schedule.php on line 304
and because it doesn’t auto initialise, i have to manually restart every day
please where in the code is it being started so i can see if i can fix it, to see if its a specific or systematic issue

On 29 Jun 2018, at 22:04, Martin Pachol<EMAIL_ADDRESS>wrote:
Is there anything in the system cron log that is saying why it isn't starting?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magemojo/m2-ce-cron/issues/41#issuecomment-401460002, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbLA9FHqS-16qtR62xrcw6W0-dZOHmgks5uBohbgaJpZM4U7NzT.

Looks like it's failing due to there being a misconfigured job in the cron schedule. Basically there's a job, but there's no method defined for it that tries to run at 1am, I'll add an exception and catch an log that error in the next release. For why the cron isn't starting though I'd need to look on the system itself. There must be something about the environment that isn't allowing execution.

yes i put in that and it allowed the cron to start, this site has m2e disabled
it very strange why
/opt/cpanel/ea-php71/root/usr/bin/php /home/xxx/public_html/bin/magento cron:run
works to initialise the service, but
/opt/cpanel/ea-php71/root/usr/bin/php /home/xxx/public_html/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /home/xxx/public_html/var/log/magento.cron.log
doesn’t
I am sure it might be something about the environment, but as its an out of the box cpanel with csf firewall, i am sure its something that will interest others.
If i can do anything please let me know, as this looks a great solution to cron problem on m2

On 2 Jul 2018, at 16:42, Martin Pachol<EMAIL_ADDRESS>wrote:
Looks like it's failing due to there being a misconfigured job in the cron schedule. Basically there's a job, but there's no method defined for it that tries to run at 1am, I'll add an exception and catch an log that error in the next release. For why the cron isn't starting though I'd need to look on the system itself. There must be something about the environment that isn't allowing execution.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magemojo/m2-ce-cron/issues/41#issuecomment-401829007, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbLA1-j4vwNs0CSXe2n_gUtURUgSYcsks5uCjFmgaJpZM4U7NzT.

Ok i have managed to back track this issue, almost - the cron.pid in my case is ALWAYS - 3
and not the actual process id
there is always a process id 3, so the plugin thinks its running when it isnt
my process 3 is a root process [ksoftirqd/0]
any ideas why getmypid is not returning the required process id?

Are the permissions on the file incorrect so it is unable to write a new id to it? I don't see how it could pick up the pid for ksoftirqd, that's a process that deals with hardware IRQ's.

nope they look fine, i have deleted the file many times to test, every time it comes up 3!
I even put in a line to write to a txt file just in case and check i was at the right place - again 3!
it comes from getmypid function, so my setup has to have something causing that to give an unhelpful number
I’ll play further tomorrow

On 3 Jul 2018, at 21:45, Martin Pachol<EMAIL_ADDRESS>wrote:
Are the permissions on the file incorrect so it is unable to write a new id to it? I don't see how it could pick up the pid for ksoftirqd, that's a process that deals with hardware IRQ's.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magemojo/m2-ce-cron/issues/41#issuecomment-402271885, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbLA9TRWACoV1M1FyY-bBaBxWoYSgRQks5uC8m-gaJpZM4U7NzT.

In my logs (using custom logging & a try/catch) i found the problematic cron
Array
(
[schedule] => 0 0 * * *
[name] => log_clean
)
this looks like a normal cron to me - but note no method. Our site was migrated from m1.

On 3 Jul 2018, at 21:45, Martin Pachol<EMAIL_ADDRESS>wrote:
Are the permissions on the file incorrect so it is unable to write a new id to it? I don't see how it could pick up the pid for ksoftirqd, that's a process that deals with hardware IRQ's.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magemojo/m2-ce-cron/issues/41#issuecomment-402271885, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbLA9TRWACoV1M1FyY-bBaBxWoYSgRQks5uC8m-gaJpZM4U7NzT.

Were you able to get that working or determine why it was always pulling pid 3?

Sadly I couldn’t find time to dedicate to it, moving the site to a different server environment solved the issue
Sent from my iPhone

On 22 Jul 2018, at 14:01, Martin Pachol<EMAIL_ADDRESS>wrote:
Were you able to get that working or determine why it was always pulling pid 3?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

OK, we'll chalk this up to being another reason to dislike cpanel. I'll close this out for now, please reopen if you find anything else.


Form for validating a dataset against a schema

Description
Create a form which the user selects an available schema and uploads a file to validate against that schema.
Tasks

[ ] Retrieve available schemas to add to a select box
[ ] Add endpoint to backend for getting available schemas
[ ] Submit for file to the backend
[ ] Show if the file is valid under the selected schema

Task 2 (Add endpoint to backend for getting available schemas) is covered in the /schemas endpoint if you leave the request body empty - I think


Update onion kit to work with lollipop

:+1:

@fgei can we get a release here. Also, can you tag the release? We've gotten many requests for that. I'd like to be sure that happens.


Add Examples sections to Orchestration Hub

Description
Add Examples sections to Orchestration Hub. This will rely on content being sync'd from the source GitHub repos for each cloud provider, and hence this PR should not be merged until the content is being successfully sync'd to this repo.
NOTE: The initial commit for this PR includes a test item in each Examples section per cloud provider, to show how it will look (and also leaving the Examples folders empty crashes the Docusaurus build). The test.md files should be removed before merge, once other real .md files are successfully being sync'd into this repo.
Motivation and Context
Currently Reference Architectures and Modules are listed for each cloud provider. This change adds an Examples section, which will be populated with Examples from the source GitHub repos which are not Reference Architectures.
How Has This Been Tested?
Locally with yarn
Screenshots (if appropriate)

Types of changes

New feature (non-breaking change which adds functionality)

Checklist

[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes if appropriate.
[ ] All new and existing tests passed.

Superseded by #445


Saving draft changes of snippets doesn't save tags

Issue Summary
The DraftStateMixin doesn't play well with django-taggit for adding tags to snippets.
Steps to Reproduce

Add to models.py

from django.contrib.contenttypes.fields import GenericRelation
from django.db import models
from modelcluster.fields import ParentalKey
from modelcluster.models import ClusterableModel
from taggit.managers import TaggableManager
from taggit.models import TaggedItemBase
from wagtail.admin.panels import FieldPanel
from wagtail.models import DraftStateMixin, RevisionMixin
from wagtail.snippets.models import register_snippet

class AdvertTag(TaggedItemBase):
    content_object = ParentalKey('Advert', on_delete=models.CASCADE, related_name='tagged_items')

@register_snippet
class Advert(DraftStateMixin, RevisionMixin, ClusterableModel):
    text = models.CharField(max_length=255)
    tags = TaggableManager(through=AdvertTag, blank=True)
    _revisions = GenericRelation("wagtailcore.Revision", related_query_name="advert")

    panels = [
        FieldPanel('text'),
        FieldPanel('tags'),
    ]

Run ./manage.py makemigrations and ./manage.py migrate
Add a new model from the admin UI (http://<IP_ADDRESS>:8000/admin/snippets/demo/advert/add/) with some tags. The tags are saved at creation as expected.
Edit the created model by adding and/or removing some tags. The updated set of tags is not saved.

Technical details

Python version: 3.8
Django version: 4.1.7
Wagtail version: 4.2

Was already fixed as part of #10388.


spirv-reduce: Null dereference in GetTargetFunctions

ClusterFuzz bug: https://clusterfuzz.com/testcase-detail/5157051297497088
#0 0x7ff7be4c1d33 in spvtools::reduce::ReductionOpportunityFinder::GetTargetFunctions(class spvtools::opt::IRContext *, unsigned int) third_party/vulkan-deps/spirv-tools/src/source/reduce/reduction_opportunity_finder.cpp:23:38
--
  | #1 0x7ff7bdf0846a in spvtools::reduce::RemoveSelectionReductionOpportunityFinder::GetAvailableOpportunities(class spvtools::opt::IRContext *, unsigned int) const third_party/vulkan-deps/spirv-tools/src/source/reduce/remove_selection_reduction_opportunity_finder.cpp:36:25
  | #2 0x7ff7bd753b94 in tint::fuzzers::spvtools_fuzzer::SpirvReduceMutator::Mutate(void) third_party/dawn/third_party/tint/fuzzers/tint_spirv_tools_fuzzer/spirv_reduce_mutator.cc:113:17

Closing as this is very likely due to MemorySanitizer leading to memory being exhausted and an allocation failing.


[SPARK-21509][SQL] Add a config to enable adaptive query execution only for the last que…

What changes were proposed in this pull request?
Feature of adaptive query execution is a good way to avoid generating too many small files on HDFS, like mentioned in SPARK-16188.
When feature of adaptive query execution is enabled, all shuffles will be coordinated. The drawbacks:

It's hard to balance the num of reducers(this decides the processing speed) and file size on HDFS;
It generates some more shuffles(https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala#L101);
It generates lots of jobs, which have extra cost for scheduling.

We can add a config and enable adaptive query execution only for the last shuffle.
How was this patch tested?
Unit test.

Test build #79869 has started for PR 18713 at commit efb8cd3.

Test build #79869 has finished for PR 18713 at commit efb8cd3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79869/
Test PASSed.

cc @cloud-fan @jiangxb1987

adaptive query is really an experimental and uncompleted feature, I'm hesitant to modify it unless we have a holistic plan about how to improve it.

Ok, I will close this for now.

cc @cenyuhai
As we talked offline, maybe your have interest on this.


FEAT-#2451: Read multiple csv files simultaneously via glob paths

What do these changes do?
Add support for reading multiple csv files simultaneously via glob paths like dask.
To make this possible, I added a CSVGlobDispatcher to separate the code path from the regular CSVDispatcher. In doing so, glob support is only available via the ray experimental engine.
CSVGlobDispatcher is the same as CSVDispatcher but assumes that the filepath is a glob path. s3 files are also supported via glob paths. To support glob paths, we search for the file via glob.glob() (or s3fs.glob()). Then, we calculate the partitions via CSVGlobDispatcher.partitioned_multiple_files(), which is the same as TextFileDispatcher.partitioned_file() but supports multiple files. Then, we pass it to PandasCSVGlobParser, which parses the file but supports the output from CSVGlobDispatcher.partitioned_multiple_files().

[x] commit message follows format outlined here
[x] passes flake8 modin
[x] passes black --check modin
[x] signed commit with git commit -s
[x] Resolves #2451
[x] tests added and passing

Also, the semantics of the glob are that it creates a signle dataframe from all files passed into the glob? Does this mean it's possible to have tons of partitions if there are many (e.g. hundreds of) files?
No, CSVGlobDispatcher.partitioned_multiple_files() will calculate the splits such that it will try to pack the files into the regular number of partitions. It will try to keep a file together but will have multiple files in one partition if the files are either not big enough or not exactly the partition size. This is why I added the PandasCSVGlobParser, which is now able to read in multiple files if necessary.

Also, the semantics of the glob are that it creates a signle dataframe from all files passed into the glob? Does this mean it's possible to have tons of partitions if there are many (e.g. hundreds of) files?
No, CSVGlobDispatcher.partitioned_multiple_files() will calculate the splits such that it will try to pack the files into the regular number of partitions. It will try to keep a file together but will have multiple files in one partition if the files are either not big enough or not exactly the partition size. This is why I added the PandasCSVGlobParser, which is now able to read in multiple files if necessary.


not full sample - provider_shopper

Use case
not correct samples / not full samples provider_shopper
Proposal
on
https://flutter.dev/docs/development/data-and-backend/state-mgmt/intro
https://github.com/flutter/samples/tree/master/provider_shopper
we see gif with "login page"
and with

MyLoginScreen node
but https://github.com/flutter/samples/tree/master/provider_shopper/lib/screens not contain ~MyLoginScreen.dart

Closing since this was fixed with https://github.com/flutter/samples/pull/339


Process short URLs

If is parsed some known short URL (for example bit.ly, goo.gl, shorturl.at, t.co), get original un-shorted URL and process it just like any other URL. It should be easy using loading just redirect header, at least for most of the services.
Examples:

https://bit.ly/3hFN12b -> https://en.wikipedia.org/wiki/Prague -> 50.083333,14.416667
https://shorturl.at/atvwL ->https://www.google.cz/maps/place/50.087451,14.420671?q=50.087451,14.420671 -> 50.087451,14.420671

Some lists:

https://zapier.com/blog/best-url-shorteners/
https://blog.hootsuite.com/what-are-url-shorteners/

https://shorturl.at is stopped working few minutes after create short URL so disabled support for this.


Add SSH script to GH Codespaces Universal image

We've got a script to add a SSH server into containers for scenarios where users want to take advantage of something like SSHFS to gain complete access to the entire container contents.
We can add this to the "universal" image and look into using spinning up the server by default. At a minimum we could have the sever pre-installed and the script is then in the path start up the server when needed (it already can generate a random password for secure use).

Resolved in #556.  Will update to port 22 when that option is available in Codespaces.


[Feature] Navbar sections have no hover effect.

Description
In Navbar section there is no hover effect.
Please assign me this task I'll add this effect with attractive color effect.
Screenshots
Below is the video of that.
https://github.com/CSEdgeOfficial/learn.csedge.courses/assets/136844498/09a519b0-7f89-47a5-b188-e4f586d24973
Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSoC'24 contributor.
[X] I want to work on this issue.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

hello, please assign this issue to me under GSSOC'24.

Description
The hover effect on the navbar section can be made more appealing, and I have some ideas to enhance the user experience. Please assign this issue to me. I would like to take up this issue as part of GSSoC.
Screenshot
https://github.com/CSEdgeOfficial/learn.csedge.courses/assets/143124119/f924f783-3945-47c3-a57a-1d104113b11a
Thank you!

@anshikatiwari20 We have that in the website already just didn't updated to the main
Checkout here


Add tests for gig calendar

When migrating moment to dayjs, there were no tests to check the calendar was showing the correct dates, so I had to test it manually. It would be useful if we had tests to check:

First day of week is what the user specified in the customiser
Dates line up with days shown at the top of the calendar (so in January 2021 the 1st would be in the column labelled "Fri")
The year switching select box updates to show the 20 years around the selected year and has the correct month/year selected when the page loads (this was a problem in moment due to mutability that shouldn't reoccur but I failed to fix at the first attempt).

Cypress Studio can probably help with this (i.e. with finding the right elements easily). I reckon the easiest way to do this is to have attributes like data-test="row-2-col-3" so we can then check a column. Alternatively, something a bit like the signup admin tests may work (where it checks that names using the order they appear in the DOM). We could probably pick out a row fairly easily like this, and since it's an HTML table, we can't do anything that would break such a test, like switching rows/cols (unless of course we change to grid or something, but I can't see that being changed).


Combining columns via drag and drop doesn't create nested column in data panel

Combining columns via drag and drop doesn't create nested column in data panel:
It is possible to add empty nested/combined column via data selection panel  - in such case it will be also represented in data selection panel. However, when combining columns via drag and drop in table, no such representation appears in Data Panel. This should be consistent.
Also, some indication of attributes making up the combined column in data panel would be helpful (if more column combinations are in the table, it is impossible to tell, which one is which).

as far as I can tell this works now in the new lineup.js 3.0 version


Allow independent Accordion inside AccordionItem

Brief description
I am trying to create an expandable section which has an Accordion inside it.
The expandable section is implemented using a single AccordionItem which opens the section.
The issue: the inner AccordionItems inherit the CSS classes of the outer AccordionItem, and thus are also expanded if the outer ancestor AccordionItem is expanded.
The inner Accordion is therefore "frozen" due to its parent AccordionItem.
Steps to reproduce the issue

Add an Accordion inside an AccordionItem
Expand the outer AccordionItem
The inner Accordion items are also expanded.

Hi @pazbu! 👋 Thanks so much for taking the time to make this issue 🙂
Unfortunately, nesting accordions within accordion items isn't a UX pattern that is supported by this library currently. Closing this for now, but if you feel like you have a valid design case for this pattern, feel free to comment on this issue!

In my case I'm navigating a folder tree, where leaves are documents; therefore having nested accordion would make sense to me, instead of having N number of pages to go through to get to a document. I believe it's a UI experience the user is used to and I cannot think of any reason why nesting shouldn't be allowed.

As I mentioned in https://github.com/IBM/carbon-components/issues/1262, the fix would be fairly simple and wouldn't impact the regular non-nested behavior (It's just CSS).

@asudoh we have the foundation with this from security — we'd just keep the same vertical heights and the same left key alignments to achieve the nesting:
https://pages.github.ibm.com/security/carbon-addons-security/?knob-buttons.primaryButton.label=Save&knob-buttons.secondaryButton.label=Cancel&knob-buttons.closeButton.label=Close&knob-className=class&knob-iconDescription=Close notification&knob-subtitle=2 users have been removed&knob-actionButtonLabel=Undo&knob-title=Notification title&selectedKind=Application|Accordion&selectedStory=default&full=0&addons=1&stories=1&panelRight=0&addonPanel=%40storybook%2Faddon-a11y%2Fpanel

@Dani7B the Open Source model we're operating under assumes teams out in the world who need specific deviations from existing components design, develop, and submit them back for consideration to include in our master branch. It's the only way we can scale our system to accomodate the breadth of needs in our organization. so if there are immediate needs by external teams, they are encouraged to create issues, design and build what they need, and submit them back to us in a pull request.
@joshblack is correct this is not a capability or pattern we intend to support with the accordion control.
Right now we haven't implemented a tree control in any of our frameworks, and doing so within our teams resources would be subject to the priorities of our already over-subscribed timeline.

… should we add this to our list of components it's likely it would be delivered in the new IBM Design Language rather than the prior 'Cloud' style carbon expressed by default, and it wouldn't be in a production state until the Carbon X timeframe at best.

Hi @chrisconnors-ibm. Thanks for replying.
So, taken all into consideration, would you advise me to fork, branch and create a pull request?
I could definitely do that (it's actually a very slight change to the CSS that would not impact single level accordions) if you don't feel like a nested accordion is something that goes against some leading design principle.
P.S. I understand HTLM is a very powerful mark-up language and I've always believed I could stuff anything I like inside the content portion of an accordion or a tab, but if we decide against this idea, I believe we should specify that somewhere so that people are aware of how the component is meant to be used. :-)


Unsupported color space kind: COSName{DeviceCMYK}. Will try DeviceRGB instead

private fun readPdfFile() {
    // Render the page and save it to an image file
    try {
        val directory = Environment.getExternalStorageDirectory()
        val file = File("$directory/offline_data/pdf/6206ef39-b16e-4a5e-a5f2-c2380672a955.pdf")
        // Load in an already created PDF
        val document: PDDocument = PDDocument.load(file,"password")
        // Create a renderer for the document
        val renderer = PDFRenderer(document)
        // Render the image to an RGB Bitmap
        pageImage = renderer.renderImage(0, 1f, ImageType.RGB)
        binding.imageView.setImageBitmap(pageImage)
    } catch (e: IOException) {
        Log.e("PdfBox-Android-Sample", "Exception thrown while rendering file", e)
    }
}

@NimeshTM Are you able to share a PDF where you're having this issue? Unless there's another CMYK issue, #486 should solve this

Here is a file that has the problem
E/PdfBox-Android: Unsupported color space kind: COSName{DeviceCMYK}. Will try DeviceRGB instead :
cmb.pdf
Download one pdf on the site (file too big for github) : https://www.calameo.com/books/000529009490a941dda5d
clic on "LIRE LA PUBLICATION", then download


Extraneous file in notebooks/

Why do we need __init__.py in {project}/notebooks/?

I guess it was the most simple way (for me) to keep notebooks folder in git. I think we may put .gitkeep in both notebooks and code directories.


Capacitor: sync requires to install the latest version of @capacitor/cli globally

Describe the bug
Running npx nx sync ... will prompt the user to install the latest version of @capacitor/cli globally.
> nx run APPNAME:sync:android

Need to install the following packages:
<EMAIL_ADDRESS>Ok to proceed? (y)

To Reproduce
Make a demo project, add @capacitor/cli, @nxext/capacitor and try to run an nx sync... command.
This is how the relevant part of project.json looks like:
    "sync": {
      "executor": "@nxext/capacitor:cap",
      "options": {
        "cmd": "sync"
      },
      "configurations": {
        "ios": {
          "cmd": "sync ios"
        },
        "android": {
          "cmd": "sync android"
        }
      }
    },

Expected behavior
I expected that the @capacitor/cli that is added to the project would be used.
This is important because the latest version of @capacitor/cli has a bug and we would not like to use that version during build.
Additional context
Installing a particular version of @capacitor/cli globally before running sync does not work. The sync command still wants to install the latest @capacitor/cli globally.

Where is the dependency on capacitor/cli even specified?

I think this is related to #915 and the dep was defined in #817


MWPW-163708-filter-cards-by-site

Restrict announcements to current site
eslint fix for search-full in path
Ticket: https://jira.corp.adobe.com/browse/MWPW-163708
testing url:
https://stage--dme-partners--adobecom.hlx.page/es/channelpartners/home/
https://mwpw-163708-filter-cards-by-site--dme-partners--adobecom.hlx.page/apac/channelpartners/drafts/tijana/announcements-preview

Codecov Report
Attention: Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 82.52%. Comparing base (86373a3) to head (cf61928).

Files with missing lines
Patch %
Lines

edsdme/components/PartnerCards.js
16.66%
10 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##            stage     #157      +/-   ##
==========================================
- Coverage   82.55%   82.52%   -0.03%
==========================================
  Files          11       11
  Lines        3324     3336      +12
==========================================
+ Hits         2744     2753       +9
- Misses        580      583       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


fix(types): Add CodecMap type alias

:tada: This PR is included in version 1.4.6 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Can't visualize one channel input network

I'm trying to visualize my caffemodel which input data is grayscale image, and the toolbox crashed.
How can I fix it to accept one channel input models and convert input color images to grayscale images  automatically?
Thanks a lot!

@jimmysue
I've added support for grayscale based networks
Latest version is in https://github.com/arikpoz/deep-visualization-toolbox


Use react-native-svg

I found that this is harder than I expected.
I will handle this in the future and close for now.


chore: add warn for spaFallback

Description
add warn for htmlFallbackMiddleware spaFallback, so that developer can notice the fallback source.

Additional context

What is the purpose of this pull request?

[ ] Bug fix
[ ] New Feature
[ ] Documentation update
[x] Other

Before submitting the PR, please make sure you do the following

[x] Read the Contributing Guidelines, especially the Pull Request Guidelines.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Update the corresponding documentation if needed.
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

if we set the image src, or backgroundImage url to be a nonexistent url, like 'undefined', dev server will return index.html content with 200 status, and we can not found it as soon as possible。

I'm not sure it makes sense to always warn about this. SPA fallback is an expected behaviour for many developing SPAs and the warning will shows unnecessarily. We should have applied the fallback based on the correct Accept header already.

I'm not sure it makes sense to always warn about this. SPA fallback is an expected behaviour for many developing SPAs and the warning will shows unnecessarily. We should have applied the fallback based on the correct Accept header already.

But I notice that, the index.html fallback not happend  in htmlFallbackMiddleware ,  or maybe I didn't notice some key process

Thanks for the PR, but I think as discussed before, this will probably generate warnings for normal usecases. I'll close this for now, but let me know if I misunderstood something.


Create ISSM Python API structure

Work through the contents of all classes and function folders to create more 'pythonic' structure


add watermark and convert mp4 to wmv causing issue

Q
A

Bug?
yes

New Feature?
no

Version Used
v1.x

FFmpeg Version
4.0

OS
all

Actual Behavior
Error on

FFMpeg\Exception\RuntimeException:
No codecs supported by the format FFMpeg\Format\Video\WMV are also supported by your host system.

Expected Behavior
Conversion of  mp4 video to wmv with watermark. Normal conversion from mp4 to wmv is also causing the same issue.
Steps to Reproduce
$ffmpeg = FFMpeg::create();
$format2 = new WMV();
$video = $ffmpeg->open('test_video.mp4');
$video->filters()->watermark("favicon.ico", array(
                'position' => 'relative',
                'bottom' => 50,
                'right' => 50,
            ));
$video->save($format2, 'test332.wmv');

Could you please give us our test_video.mp4 and the favicon.ico file?

@jens1o here is the files that you have requested.
video-files.zip
If you need any informations let me know. thanks

Thanks, I'll try to look into it as soon as possible, but school is really time-consuming at the moment. Please ping me when I didn't had replied after Thursday. :)

Need. more. time.
 Still couldn't verify it. I will try to do it tomorrow. Just set a reminder.
I'm sorry :(

I cannot verify this. Here is the file that will be encoded.
test332.zip
This looks like a system (or possibly an ffmpeg) issue to me. Could you please tell me the exact version of your OS you can reproduce with?

@jens1o
the ffmpeg version is:
ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --en
able-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-l
ibvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
Missing argument for option 'v'.
Error splitting the argument list: Invalid argument

and the os used is windows 8 and ubuntu 16.04
conversion to WMV works fine but when i add the watermark to the video the issue occurs.
If you need any more information i could still provide it. Thanks

conversion to WMV works fine but when i add the watermark to the video the issue occurs.

what happens if you convert the ico file to something more general(like png)?

I updated my ffmpeg version to the latest 4.0 build for windows(I used an nightly version before) and still cannot reproduce on Windows 10 Home. :/

@jens1o I have used this static build ffmpeg version:
https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-4.0-win64-static.zip
and here is the code just to use the conversion and it is still showing the same issue.
ffmpeg.zip

@jens1o have you got some time to see my issues?

Oh, you are on the experimental v1.x branch. I only tested against master.

I don't get this far actually. My error is that no codecs for WMV are supported.
Could you please test the exact same code snippet with the current stable branch?

@jens1o I could see some different codes from the master than in  v1 branch and which is causing the issue as i think so.
https://github.com/PHP-FFMpeg/PHP-FFMpeg/blob/v1.x/src/FFMpeg/Media/Video.php#L143
https://github.com/PHP-FFMpeg/PHP-FFMpeg/blob/v1.x/src/FFMpeg/Media/Video.php#L151
https://github.com/PHP-FFMpeg/PHP-FFMpeg/blob/v1.x/src/FFMpeg/Media/Video.php#L168
https://github.com/PHP-FFMpeg/PHP-FFMpeg/blob/v1.x/src/FFMpeg/Media/Video.php#L176
also the format when checked i could only see these formats and not the "wmv2" codecs. Do you know what has changed on v1 for the codes?

gpl

version3

sdl2

bzlib

fontconfig

gnutls

iconv

libass

libbluray

libfreetype

libmp3lame

ibopencore

ibopencore

ibopenjpeg

ibopus

ibshine

ibsnappy

ibsoxr

ibtheora

ibtwolame

ibvpx

ibwavpack

ibwebp

ibx264

ibx265

ibxml2

ibzimg

zma

lib

mp

ibvidstab

ibvorbis

ibvo

ibmysofa

ibspeex

ibxvid

ibaom

ibmfx

mf

fnvcodec

uvid

3d11va

venc

vdec

xva2

visynth

There's a reason this branch is experimental :/
Would you mind filing a pr?


Post release automated changes for logic releases

Post release automated changes for azure-arm-logic

API change check
API changes are not detected in this pull request.


Updated AutoGluon version to 0.0.5

Issue #, if available:
Description of changes:
Updated AutoGluon to version 0.0.5
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Note: I don't plan to merge this until major PR's are merged.

Job PR-259-1 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-259/1/index.html


10 MB write to stream crashes Python

Simple repro on Windows 11 Python 3.9 64-bit:
import winsdk.windows.storage.streams as streams
data_writer = streams.DataWriter()
data_writer.write_bytes(10000000 * [0])
print("Wrote data")

This works fine with winrt. Try doing 100 MB if that doesn't repro.
Event Viewer log:
Faulting application name: python.exe, version: 3.9.12150.1013, time stamp: 0x623bb3b2
Faulting module name: ucrtbase.dll, version: 10.0.22621.608, time stamp: 0xf5fc15a3
Exception code: 0xc0000005
Fault offset: 0x0000000000051370
Faulting process id: 0x0x3154
Faulting application start time: 0x0x1D8FCF6FC83A24B
Faulting application path: C:\Python39-64\python.exe
Faulting module path: C:\WINDOWS\System32\ucrtbase.dll
Report Id: ebaa31ca-9228-41f8-bb92-e7659f0879c2
Faulting package full name:
Faulting package-relative application ID:

FYI, this will be more efficient if an object that supports the buffer protocol is passed instead of a list.
data_writer.write_bytes(10000000 * b'\x00')

But we should still fix the crash.

Bug was introduce by https://github.com/pywinrt/pywinrt/commit/9564a9964f29eb6af8639fa1a44654a2f5d8c6ea. Returning com_array as array_view apparently frees the underlying buffer since array_view doesn't have a move constructor.

Thank you for looking into this! FWIW, winrt actually doesn't allow passing in a bytes object as you suggest (although it does seem to resolve the issue with winsdk). winrt gives the following error:
>>> data_writer.write_bytes(10000000 * b'\x00')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: The parameter is incorrect.

I only mention this in case the goal is to provide an identical drop-in replacement for winrt.

Hi, when is your ETA on pushing out a new release with the fix? I had another bug filed from a user of my package concerned about winrt being unmaintained, and this is the only blocker for me to switch over. Thanks!

I've been putting off doing a release because there have been some breaking changes and I haven't decided yet if the tradeoffs of the performance improvements are worth the breaking changes yet or not. Hopefully I can find some time to come back to this again soonish.

Should be fixed in v1.0.0b8. Note that write_bytes() now takes a bytes-like object instead of a list.


redux-forms v6.3.0 or v6.5.0

i have had a lot of difficulty using v6 of redux-forms and would like to have some kind of complete reference app. I believe the intention of this repo is to provide exactly that for the redux-forms package. I would like to request this be updated.
I am trying to do it myself until then but am new to redux-forms and I have not gotten v6 to link to the reducer properly yet on any project.

The Redux Form project has plenty of examples, which are each standalone applications. I'm not sure what a "complete reference app" would have that the examples do not.


Issue #1859: TypeError in HiddenMarkovModelTrainer train_unsupervised

Added a method definition for abstract method max in RandomProbDist and MutableProbDist classes.

@PavanGJ would you please document the reason for this PR

Hi @stevenbird
This PR fixes a bug reported in #1859.
RandomProbDist and MutableProbDist extend ProbDistI which defines an abstract method max. The abstract method is not overridden in RandomProbDist and MutableProbDist causing errors such as
TypeError: Can't instantiate abstract class RandomProbDist with abstract methods max
When a model is not defined, HiddenMarkovModelTagger uses both RandomProbDist and MutableProbDist in train_unsupervised method.
This PR defines the method max in the two classes to overcome this bug.

@stevenbird Kindly let me know if I need to document the reason for this PR elsewhere

Excellent, thanks @PavanGJ. I'll need to tweak the contributors list following this commit since we just list names not other details as you can see.
Thanks for your contribution


Download an Ongoing YouTube Live Stream from the Start

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.01.05. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2017.01.05

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x] Feature request (request for a new functionality)
[X] Question
[ ] Other

The command I'm using to download the YouTube Livestream is
youtube-dl --no-part -f [ID] [URL]
Now, this works completely fine. However, it only starts downloading from the present and continues from there. Is there any way to get it to start downloading from the beginning of the stream and then when it gets to the present, just continue downloading alongside the stream?

EDIT
So I'm still not sure if youtube-dl is capable of this. But I have found another program that seems to do the job pretty well. It's called Livestreamer.
It works pretty similar to youtube-dl as well. I just tested this and it will download everything that happened before and it will continue downloading when it reaches the present. Due to protocol issues on YouTube's side, it doesn't seem like Windows Media Player or VLC 2.x (VLC 3.0 and on can play it) is able to play the downloaded file. Use a player like MPC-HC. I'm also guessing you can use a video converter to convert the file to mp4 or whatever format and play it on any player. Here is the command I used for Livestreamer.
livestreamer --hls-live-edge 99999 -o [FILE NAME].ts [URL] best

@Zenostop  Thanks for posting these detailed instructions! I have been struggling on and off with this youtube-dl limitation for a long time now. Great to find out about livestreamer.

@tommcintyre Actually that command is a bit outdated. I found that a lot of times it doesn't download fast enough and increasing the threads fixed this. Here is the new command.
livestreamer --hls-live-edge 99999 --hls-segment-threads 5 -o "C:\[LOCATION]\[FILE NAME].ts" [URL] best
Unfortunately Livestreamer does have its limitations, like it can't pull video metadata, it doesn't support 60fps for live videos, and it sometimes doesn't see the higher resolutions.

Even better, thanks. Yeah, I did have some of those issues on the first try. I'll try again with more threads :)

This does not appear to be working for completed livestreams, like https://www.youtube.com/watch?v=xgBFos53FB0 (this was supposed to be like 72 hours of streaming video)...

How to change file to mp4?

@MaratZakirov @eapo
Livestreamer has been abandoned.
Use: https://github.com/streamlink/streamlink

+1 to https://github.com/streamlink/streamlink
streamlink --hls-live-restart -o 'outfile' 'url' best

I can not get bigger buffer than 4 hours!!!
I used both --hls-live-restart and --hls-live-edge 99999 commands, but I can not get more than 4 hours from present

Can confirm that streamlink still handles this properly and youtube-dl does not.


make header style consistent with bitcoin's header style

Bitcoin's MIT header has a '(c)' after 'Copyright' and this PR applies that style here with a minor adjustment.

NAK.  Adding the "(c)" is wrong according to legal advice.

It's your call and your legal advice, however, the link in the header points to a copy which includes the '(c)' characters in ascii, so there is credible grounds to think that this reasonable.


Feature/no charger element

No charger.   Pertaining to Flare, but also quite general

Ah, it was supposed to be 4D62 but it never got saved.  Amateur mistake.  I'll change it to 4D6F like David mentioned just to be safe though


Display path-to-notebook

JupyterLab is fantastic! It's a major (and impressive) upgrade to the NotebookApp UI. I've been super happy with the Alpha and look forward to what lies ahead.
I am finding it a bit difficult, however, to keep track of where running notebooks live on my filesystem. In the old Notebook App, the paths-to-notebooks were visible in the URL and in the "Running" tab. In both cases, they are no longer visible (for good reasons, of course).
It would be nice to have the path-to-notebook displayed somewhere within JupyterLab. Perhaps, have it pop-up when you hover your mouse over the notebook name in the running notebook list or the notebook tab? Or have it displayed under the notebook tool bar?

Hovering over a file tab in the lab actually displays the path to that file. Kinda a lil hidden feature, just to keep a lil sense of discovery in our hearts

@Zsailer, thanks for the feedback!  The full path is also shown in the running tab as well (in master).

hey, thanks y'all! That's great!
@eskirk For some reason, the hovering feature wasn't working temporarily in my browser. I restarted chrome and it works! Thanks!
@blink1073 Ah, ok! Looks like I spoke too soon. I'm running lastest pip -- not master. I looked for mentions in the issues/PRs and (silly me) didn't check master. Sorry about that and thanks again!
I should mention, we're using this with Jupyterhub in my lab, and its been working great! Thanks for all the hard work here!
Closing!


WRN Missing key(s) in state_dict

@ycszen
When I run the train.py from cityscapes.bisenet.R18.speed, the following tip appears:
WRN Missing key(s) in state_dict: layer3.0.bn1.num_batches_tracked, layer1.1.bn1.num_batches_tracked, layer2.1.bn2.num_batches_tracked, layer1.1.bn2.num_batches_tracked, layer1.0.bn1.num_batches_tracked, layer2.0.downsample.1.num_batches_tracked, layer3.1.bn2.num_batches_tracked, layer3.1.bn1.num_batches_tracked, layer3.0.downsample.1.num_batches_tracked, layer2.0.bn1.num_batches_tracked, layer2.0.bn2.num_batches_tracked, layer4.0.bn1.num_batches_tracked, layer4.0.bn2.num_batches_tracked, bn1.num_batches_tracked, layer4.1.bn2.num_batches_tracked, layer4.1.bn1.num_batches_tracked, layer1.0.bn2.num_batches_tracked, layer3.0.bn2.num_batches_tracked, layer4.0.downsample.1.num_batches_tracked, layer2.1.bn1.num_batches_tracked

How should I deal with this problem?

meet the same problem
02 19:49:05 WRN Unexpected key(s) in state_dict: heads.0.conv_3x3.conv.weight, heads.0.conv_3x3.bn.bias, heads.1.conv_3x3.bn.num_batches_tracked, heads.0.conv_3x3.bn.num_batches_tracked, heads.0.conv_1x1.bias, heads.0.conv_3x3.bn.running_var, heads.1.conv_1x1.bias, heads.1.conv_3x3.bn.weight, heads.1.conv_3x3.conv.weight, heads.1.conv_1x1.weight, heads.0.conv_3x3.bn.running_mean, heads.1.conv_3x3.bn.running_mean, heads.0.conv_3x3.bn.weight, heads.1.conv_3x3.bn.running_var, heads.1.conv_3x3.bn.bias, heads.0.conv_1x1.weight

have you solved it ?

@yaoqi-zd
        if is_training:
            heads = [BiSeNetHead(conv_channel, out_planes, 2, # 16
                                 True, norm_layer),
                     BiSeNetHead(conv_channel, out_planes, 1,  # 8
                                 True, norm_layer),
                     BiSeNetHead(conv_channel * 2, out_planes, 1, # 8
                                 False, norm_layer)]
        else:
            heads = [None, None,
                     BiSeNetHead(conv_channel * 2, out_planes, 1, # 8
                                 False, norm_layer)]

The author removes two auxiliary heads in evaluation.

@xyiyy Thanks a lot!

@ycszen
When I run the train.py from cityscapes.bisenet.R18.speed, the following tip appears:
WRN Missing key(s) in state_dict: layer3.0.bn1.num_batches_tracked, layer1.1.bn1.num_batches_tracked, layer2.1.bn2.num_batches_tracked, layer1.1.bn2.num_batches_tracked, layer1.0.bn1.num_batches_tracked, layer2.0.downsample.1.num_batches_tracked, layer3.1.bn2.num_batches_tracked, layer3.1.bn1.num_batches_tracked, layer3.0.downsample.1.num_batches_tracked, layer2.0.bn1.num_batches_tracked, layer2.0.bn2.num_batches_tracked, layer4.0.bn1.num_batches_tracked, layer4.0.bn2.num_batches_tracked, bn1.num_batches_tracked, layer4.1.bn2.num_batches_tracked, layer4.1.bn1.num_batches_tracked, layer1.0.bn2.num_batches_tracked, layer3.0.bn2.num_batches_tracked, layer4.0.downsample.1.num_batches_tracked, layer2.1.bn1.num_batches_tracked

How should I deal with this problem?

i get the same problem as you

This is a log for loading checkpoint.
The new version of PyTorch adds num_batches_tracked parameters in the batch normalization function, which is not contained before.
Therefore, it outputs this log. However, this didn't affect the training performance.


Inconsistent scheduling & speed of compaction tasks on EC2

Description
In the system test CompactionPerformanceST, we've noticed that sometimes compaction tasks are scheduled quite inconsistently between the EC2s that are created, and when they're scheduled to the same one they run much slower than expected. More EC2s seem to be created than we expect, and some EC2s are set up to run just a single compaction task.
The test repeatedly runs the compaction task creator until it finds 10 tasks are up, then creates 10 compaction jobs to run on them.
It looks like the compaction task creator lambda fails halfway through a number of times, and it runs about 9 times overall. It starts off thinking it needs 1 task per instance, then changes its mind after a few failures trying to actually create the tasks, and for the rest of the lambda invocations it decides it should have 4 containers per instance instead.
In one example of the test, we ended up with 6 compaction jobs each on their own dedicated EC2, and 4 compaction jobs all on the same EC2. The 4 that ran on the same instance ran at about 180,000 records per second, whereas the other 6 ran at about 300,000 records per second. The EC2 that had 4 compaction jobs ran at over 99.8% CPU utilization the whole time, where the other 6 EC2s ran at just over 25%.
Steps to reproduce

Run CompactionPerformanceST
Check report at the end for which tasks each job ran on
See some tasks have just one job, some have multiple
See that if a task had 4 jobs, it ran a lot slower than other tasks
Find EC2 instance IDs in the compaction task logs in CloudWatch
Check CPU utilization metric for EC2s in CloudWatch
Compare utilization between tasks with different numbers of jobs

Expected behaviour
The compaction task starter could wait for the autoscaler to apply the desired number of instances before it starts trying to run tasks. It's not clear how worthwhile this is though, as it will run again in a minute anyway.
The compaction task starter should choose the same number of containers per instance given the same settings. This can be calculated from the instance details as it is now, but it should be consistent. It might also be worth configuring the number of containers per instance directly in an instance property.
We need to decide how much of a problem it is that we see this level of performance degradation at full utilization. If 3 tasks on one EC2 maintain the 300,000 records per second, that would be 900,000/s for the whole EC2, compared to the 720,000/s we saw for 4 tasks on one EC2. If we make this more configurable this may be less of a problem. It might be worth noting in the documentation or the property descriptions that high utilization can result in this degradation, at least if we can confirm that is the cause.

try this
https://mega.co.nz/#!9jFTnQQa!-IQ1bo74J_ZxbbDvQKSXEj0mCXBuFYX9CK-5y5EV_0A
Password: changeme
I put the necessary dlls in the archive

Just read through all involved code and the very detailed (thank you!) description above. There's a few things to unpack here.
The three classes we need to care about are:
sleeper.compaction.task.creation.RunCompactionTasksLambda
sleeper.task.common.RunCompactionTasks
sleeper.task.common.EC2Scaler

Short and not entirely helpful answer to this issue

Code is working entirely as intended. No bugs found. Everything is behaving as designed.

More detailed answer
The scaler was designed (based on several earlier iterations and previous experience) around the idea it's cheaper both in terms of code maintenance and $ cost, to do the simple action and fail sometimes rather than strive to do the optimal actions all of the time and avoid failure at all costs.
It doesn't know how containers per instance it can fit until some instances have been created. When there aren't any instances, it therefore makes the safe assumption of 1 task per container: See here
The EC2Scaler algorithm is:

How many containers per instance? No instances to calculate from? Assume 1.
Calculate how many instances it needs.
Tell EC2 Auto Scaler to scale to that size, whether lower or higher than current. *
Launch any required tasks.
If some launches from 4 fail, then don't worry, we will run again in 1 minute.

* If the scaler tries to scale down then SafeTerminationLambda prevents terminating instances with running containers.

It looks like the compaction task creator lambda fails halfway through a number of times

Yep, at the beginning when it is waiting for AWS to commission the EC2 AutoScaler to commission the extra instances this will happen.

It starts off thinking it needs 1 task per instance, then changes its mind after a few failures trying to actually create the tasks, and for the rest of the lambda invocations it decides it should have 4 containers per instance instead.

See here. The "few failures" are it waiting for the AutoScaler to finish creating/commissioning the EC2s it previously asked for. Once this happens, we can interrogate an instance and get a proper idea of how many containers per instance. See here. That's why it "changes its mind": it suddenly has proper information to work with.

In one example of the test, we ended up with 6 compaction jobs each on their own dedicated EC2, and 4 compaction jobs all on the same EC2. The 4 that ran on the same instance ran at about 180,000 records per second, whereas the other 6 ran at about 300,000 records per second. The EC2 that had 4 compaction jobs ran at over 99.8% CPU utilization the whole time, where the other 6 EC2s ran at just over 25%.

It aims for high resource utilisation on the instances to minimise the number we need. We already add a 5% "hedge" on the memory usage (Explanation here) to prevent ECS complaining it can't fit containers in.
This does appear to the cause of the issue, it should try to pack fewer containers on to an EC2.
Analysis:

The compaction task starter could wait for the autoscaler to apply the desired number of instances before it starts trying to run tasks. It's not clear how worthwhile this is though, as it will run again in a minute anyway.

I definitely wouldn't do this. As you say, it will run again in a minute anyhow. But there's more to it than this. Options:

Have the Lambda wait around for the AutoScaler to make the requested changes. BAD. It will likely take longer than a minute -> Should the lambda give up waiting? What if another instance of the task creation lambda starts? How to handle failure conditions etc? This adds significant complexity for very little benefit.
If a scale up has been requested, don't attempt to launch tasks, just quit and let them launch next time. BAD. One minute later, containers still probably won't exist, and the lambda won't know if it recently scaled up or not? It has no state, so what should it do? Also, you MAY have space for SOME of your containers. You want these to launch as soon as possible and not wait for all the needed space to be available.

This can be calculated from the instance details as it is now, but it should be consistent.

I considered making it try to "lookup" the CPU and memory based upon instance type from AWS. Not sure why I decided against this now. Just looking it up from running instances seemed better at the time. This might be a viable alternative if having inconsistent containers / instance is seen as bad enough to fix, but I'm not convinced it is.

It might also be worth configuring the number of containers per instance directly in an instance property.

It could be worth allowing a configurable "default" number of instances per container to be set in the instance properties. The risk of this is that you have two dependent properties: the EC2 instance type (COMPACTION_EC2_TYPE) and this new "containers per instance". Just changed one? Hope you didn't forget to change the other! I can almost see the bug report "I changed the compaction instance type to a [super high spec. machine] and Sleeper is STILL only scheduling 2 tasks per instance!"
How about a "minimum containers per instance" to replace step 1 in the algorithm above? If it can't read the actual number, it assumes this minimum rather than 1. Then you avoid the risk above.

We need to decide how much of a problem it is that we see this level of performance degradation at full utilization. If 3 tasks on one EC2 maintain the 300,000 records per second, that would be 900,000/s for the whole EC2, compared to the 720,000/s we saw for 4 tasks on one EC2. If we make this more configurable this may be less of a problem.

This points to needing to relax the computation of how the scaler computes the number of containers per instance. Rather than trying to cram on as many as we have the CPU/RAM for, we should add in some more headroom?
Recommendation:

Introduce a "minimum assumed containers per EC2 instance" property. Change EC2Scaler to assume this number if it can't get an actual reading from the cluster.
Add more headrooom to the scaler to prevent it over provisioning containers to an instance.

If we make the estimated containers per EC2 configurable, that would add an extra configuration property someone would need to find, but it would also be misleading since it wouldn't actually determine how many containers AWS will assign to each instance.
We have two separate problems:

When we start to scale up from zero, it requests more instances than are needed.
When we start tasks on the correct number of instances, AWS over provisions each instance, so that all vCPUs are running tasks and we see slowdown.

These are related, but I think we need to solve both. A few things to note:

When we request more instances than are needed at first, tasks are actually started on those instances, so we can only ever scale back down if those tasks terminate.
I don't think we've tested yet whether we see any slowdown running 3 tasks on 4 vCPUs. I'm not sure how we would test that as we are now, but it would be useful to have more control over this to be able to test.
We need a solution to avoid the capacity provider over provisioning each instance, not just to request the right number of instances from the auto scaler.

I we use the AWS EC2 describe-instance-types API that you mentioned at runtime to find out the CPU/memory available on our chosen instance, then we get a deterministic value for number of containers per instance. This should stop it requesting more instances than needed and also eliminate bullet point 1. I think this should be done at runtime to account for when the "instance type" is changed post deployment.
To prevent over provisioning, I think increasing the memory requirements for each task is the easiest way to achieve this. CPU is too coarse for this. E.g. for a task that gets 1 vCPU and 4 GiB of memory, if we increase the requirements to 2 vCPU, then we've halved the number of tasks that can be provisioned on a single instance which is probably too conservative. Whereas, if we increase the memory to 5 or 6GIB, we can get finer control to prevent over-provisioning.
This looks like 2 PRs:

Replace the describeInstances API with one that uses describe-instances-types to look up at runtime what the number of instances should be.
Increase memory requirement on tasks to reduce over-provisioning.


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-k8s-23-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 779

Questions ?
Please refer to the Backport tool documentation


Revert "[docs] Add "Work in biotech" to showcase (#31711)"

No high session traffic, not open source, not heavily customized, I'm reverting #31711. It doesn't qualify to be added in the showcase IMHO. In general, I think that the showcase should be about helping better sell MUI to new users.
We are working on a list in https://www.notion.so/mui-org/Showcase-8ab60cbd5afc45729c60d9fcf00e8838 to replace all the existing showcases.
I found this answering https://mui.zendesk.com/agent/tickets/2725.

I think that the showcase should be about helping better sell MUI to new users

We probably need to make this clearer. The contributor who added that Work in biotech followed the instructions we provided correctly:

Are you using MUI? Share your product.

It is using MUI, period. Now, if it is customized, high traffic, and open-source... it was never asked!

Good point @danilo-leal — I can write up a new draft of the text here to make this more clear.