[Bug] Missing model.onDidChangeContentFast listener

Reproducible in vscode.dev or in VS Code Desktop?

[X] Not reproducible in vscode.dev or VS Code Desktop

Reproducible in the monaco editor playground?

[ ] Not reproducible in the monaco editor playground

Monaco Editor Playground Code
// The Monaco Editor can be easily created, given an
// empty container and an options literal.
// Two members of the literal are "value" and "language".
// The editor takes the full size of its container.

const instance = monaco.editor.create(document.getElementById('container'), {
	value: "function hello() {\n\talert('Hello world!');\n}",
	language: 'javascript'
});

const model = instance.getModel()

model.onDidChangeContentFast(function(){}) // -> this throws error

Actual Behavior
I have used the onDidChangeContentFast event of the editor model in constrained editor plugin which attempts to solve this issue https://github.com/microsoft/monaco-editor/issues/953
But in one of the recent commits in textModel.ts that events listener is removed. https://github.com/microsoft/vscode/commit/a9002ffa96ef08e709bd5a680f97923e3df5fc2a#diff-fc0b9cbcda54fc52f7f1eb424b1945ca6e6c81c73c0efc18c6b2e48ec7755b02L251
Screenshot of the error in monaco editor playground :

Expected Behavior
I need an alternative for that listener. So, Is there any alternative for that listener exists?
Can I replace onDidChangeContentFast with onDidChangeContent ?
As onDidChangeContent uses this._eventEmitter.slowEvent and onDidChangeContentFast used this._eventEmitter.fastEvent , will it create any problem in my existing behaviour, if I replace those in my code?
Additional Context
I would recommend the developers to add an entry in the changelog of monaco editor saying this event listener is deprecated / removed.
I searched forITextModel.onDidChangeContentFast in the logs but couldn't find any of them.
Note:  Although the source of the issue is from vscode repo's textModel.ts, I have checked Not reproducible in vscode.dev or VS Code Desktop, because an user using vscode won't face this issue, but an user using monaco editor will be facing these problems

I think onDidChangeContentFast was not meant to be exposed as monaco API, it is an internal event. Fast just means that this event is called before the other one.
You should just use onDidChangeContent.

Hi @hediet, Thanks for the clarification.
By convention, I have refrained from using the methods inside model instance starting with an underscore assuming those are internal methods.
but  onDidChangeContentFast didn't have underscore preceding it, so I thought that it was an exposed listener.
From now on, I will use onDidChangeContent


Add iOS example

It would be nice to have iOS example (something similar to existing Android example).

I've created a super simple iOS app that plays a sine-wave on launch, and cleans up the audio device when the app isn't active
See https://github.com/mrDIMAS/tinyaudio/pull/9


Update how-to-connect-install-existing-database.md

Update SQL version from 2012 to 2019

@billmath
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Allow alternative inputs

The MVP product currently takes a Hail Matrix Table representation of the variant datatset as a starting point
Using the Hail VCF Combiner it should be a trivial change to allow this tool to take a single VCF, or list of VCFs as an alternative input
Note - all VCFs will also have to be present in the same execution environment (e.g. GCP), and the user (or system user) operating the process will need permission to read the data

Closing issue - Thanks to @vladsaveliev 's work, the starting point can be VCF or MT, and annotation will be carried out during the End-to-end run, instead of requiring annotated data to be provided


IOS PHONEGAP BUILD FAIL

when using Phonegap build service to build a ios app the following eroor message is give
"Error - The following 3rd-party plugin is causing the build to fail and may need to be updated to a newer version: heyzap-cordova"
in the log this seems of significance
"
The following build commands failed:
CompileC build/App.build/Release-iphoneos/App.build/Objects-normal/armv7/CDVBasePlugin.o App/Plugins/heyzap-cordova/CDVBasePlugin.m normal armv7 objective-c com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)
"
Any ideas what i can do to fix this or what i did that may have caused this

same problem


Uploading test results

Uploading test results for dynamic reordering.
Todos

[ ] Receive build number through the environment variable and the command line argument both

I tried to refactor this PR with asyncio for 3 hours. However, I finally agreed original thread code was better. I removed my code. Now, I understand this PR deeply, and ready to approve this PR.

@shuheiktgw Is this ready to merge?

I need to modify the test result format so give me a little bit more time! 🙏

@shuheiktgw Thank you!

Wait!

And fixed it

@shuheiktgw Thank you for fixing this conflict.
By the way, please stop force push from the next time. I can't follow what you changed.

Whoops sorry, I should have used squash and merge instead!


Use NettestImageURL var from shipyard

...to make the nettest image configurable for air-gapped deployments.

Are we thinking about backporting this? If so, I guess the version bump should be a separate commit.

Are we thinking about backporting this?

Not that I'm aware of. If so, we'd have to backport several more PRs.


telecablesat.fr.config.js - Error: Request failed with status code 403

Site
telecablesat.fr.config.js
Description
All channel failed with error: Error: Request failed with status code 403
Any idea how to fix this?
Loading 'sites/telecablesat.fr/telecablesat.fr.config.js'...
Loading 'sites/telecablesat.fr/telecablesat.fr.channels.xml'...
[1/244] telecablesat.fr - 13emeRue.fr - Sep 19, 2023 (0 programs)
  Error: Request failed with status code 403
[2/244] telecablesat.fr - 13emeRue.fr - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403
[3/244] telecablesat.fr - 3sat.de - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403
[4/244] telecablesat.fr - AB1.fr - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403
[5/244] telecablesat.fr - AlJazeeraEnglish.qa - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403
[6/244] telecablesat.fr - beINSports3France.fr - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403
[7/244] telecablesat.fr - CanalPlusCinemaFrance.fr - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403
[8/244] telecablesat.fr - Ketnet.be - Sep 20, 2023 (0 programs)
  Error: Request failed with status code 403

This error usually means that the site is blocked in your country.
Try running the script with VPN enabled.


How to see the logs of SDK, when its running its SDK Runtime

I've tried running the client app in debug mode to see if the logs from the sdk-app are being printed in the app or not, but there is no log in the console. What is the way to actually see the logs in the console from the sdk-app.
Example code snippet
@SuppressLint("Override")
  @Override
  public void initSdk(
      SandboxedSdkContext sandboxedSdkContext,
      Bundle params,
      Executor executor,
      InitSdkCallback initSdkCallback
  ) {
    Log.d("initSDk", Thread.currentThread().getName());
    executor.execute(() ->
        initSdkCallback.onInitSdkFinished(new Bundle())
    );
  }

  @SuppressLint("Override")
  @Override
  public View getView(Context windowContext, Bundle bundle) {
    Log.d("getView", Thread.currentThread().getName());
    WebView webView = new WebView(windowContext);
    webView.loadUrl("https://developer.android.com/privacy-sandbox");
    return webView;
  }

You should be able to see the logs in logcat. You might need to play around with the filter as the logs come from the com.google.android.sdksandbox package (so I just cleared my filter and searched the logs for my string)
i.e.

Are you seeing any errors in logcat?

@cshmerling thanks again, In the console of the run tab it was not showing but in the section log after removing the specific package or after adding com.google.android.sdksandbox package it's showing the SDK one


Migrations:up does not exit with success

Running migrations up does not exit with success, so the command does not end.
npx mikro-orm migrations:up results in a success print of migrates up to x, but then there is no successful exit.
To Reproduce
Steps to reproduce the behavior:

npx mikro-orm migrations:up
Upon completion you need to manually terminate

Expected behavior
Once migration:up has executed, the command should return a success exit code.

ive tried doing a command like this in docker to force exit, but because mikro-orm does not successfully exit, it never exits the container.
"command": [
"/bin/bash",
"-c",
"npx mikro-orm migration:up && echo 'pass' || echo 'fail'"
],


Problem upgrading from 5.8.2 to 6.0.2

Hi Valdis!
I'm trying to upgrade from:
  <package id="DbLocalizationProvider.AdminUI.EPiServer" version="5.8.3" targetFramework="net48" />
  <package id="DbLocalizationProvider.EPiServer" version="5.8.2" targetFramework="net48" />
  <package id="DbLocalizationProvider.EPiServer.JsResourceHandler" version="5.8.2" targetFramework="net48" />
  <package id="LocalizationProvider" version="5.8.2" targetFramework="net48" />
  <package id="LocalizationProvider.Abstractions" version="5.8.2" targetFramework="net48" />
  <package id="LocalizationProvider.AspNet" version="5.8.2" targetFramework="net48" />
  <package id="LocalizationProvider.JsResourceHandler" version="5.8.2" targetFramework="net48" />

To:
  <package id="DbLocalizationProvider.AdminUI.EPiServer" version="6.0.2" targetFramework="net48" />
  <package id="DbLocalizationProvider.EPiServer" version="6.0.2" targetFramework="net48" />
  <package id="DbLocalizationProvider.EPiServer.JsResourceHandler" version="6.0.2" targetFramework="net48" />
  <package id="LocalizationProvider" version="6.0.2" targetFramework="net48" />
  <package id="LocalizationProvider.Abstractions" version="6.0.2" targetFramework="net48" />
  <package id="LocalizationProvider.AspNet" version="6.0.2" targetFramework="net48" />
  <package id="LocalizationProvider.JsResourceHandler" version="6.0.2" targetFramework="net48" />

I get:
[KeyNotFoundException: The given key was not present in the dictionary.]
   System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key) +14392481
   DbLocalizationProvider.TypeFactory.GetHandler(Type queryType) +27
   DbLocalizationProvider.TypeFactory.GetCommandHandler(TCommand request, Type wrapperType) +281
   DbLocalizationProvider.ICommandExtensions.CanBeExecuted(ICommand command) +72
   DbLocalizationProvider.Sync.Synchronizer.UpdateStorageSchema() +31
   DbLocalizationProvider.Sync.Synchronizer.DiscoverReadMerge() +51
   DbLocalizationProvider.IAppBuilderExtensions.UseDbLocalizationProvider(IAppBuilder builder, Action`1 setup) +593
   DbLocalizationProvider.EPiServer.DbLocalizationProviderInitializationModule.SetupLocalizationProvider(Object sender, EventArgs eventArgs) +124

[TargetInvocationException: Exception has been thrown by the target of an invocation.]
   System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) +0
   System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) +132
   System.Delegate.DynamicInvokeImpl(Object[] args) +160
   EPiServer.Framework.Initialization.InitializationEngine.OnInitComplete() +263
   EPiServer.Framework.Initialization.InitializationEngine.ExecuteTransition(Boolean continueTransitions) +208
   EPiServer.Framework.Initialization.InitializationModule.EngineExecute(HostType hostType, Action`1 engineAction) +423
   EPiServer.Framework.Initialization.InitializationModule.FrameworkInitialization(HostType hostType) +226
   EPiServer.Global..ctor() +42
   StockholmParkering.Web.EPiServerApplication..ctor() +37
   ASP.global_asax..ctor() +42

[TargetInvocationException: Exception has been thrown by the target of an invocation.]
   System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck) +0
   System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark) +142
   System.Activator.CreateInstance(Type type, Boolean nonPublic) +107
   System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark) +1476
   System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes) +186
   System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture) +28
   System.Web.HttpRuntime.CreateNonPublicInstance(Type type, Object[] args) +80
   System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context) +182
   System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext) +369

[HttpException (0x80004005): Exception has been thrown by the target of an invocation.]
   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +532
   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +111
   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +724

I notice I had 5.8.3 on DbLocalizationProvider.AdminUI.EPiServer, and not the other ones, could that be an issue?
Is it a malformed translation somewhere (guess not) or a schema upgrade issue (seems so)?
How to pin-point and fix? ;)

I can't figure out exactly which repository to add this bug in, it's in the initialization in EPi calling code in https://github.com/valdisiljuconoks/LocalizationProvider and the error is thrown there. So I guess you have to look at both repositories to find a solution (or improved logging or something?).

No problems. Sometimes I think split into submodules was bad decision ;)
Anyways - please read notes, seems like you are missing SQL server init code. It will be more clean error message in upcoming 6.1 version.
Sorry for trouble

Ok. Release notes after upgrade? Or documentation of how to setup? You mean there’s a manual step or is there a configuration missing in my code?
Will try upgrade again to see if I’ve missed to read some release notes... 😉

You will need to call .UseSqlServer() on configuration context to add proper handlers and tell provider that it needs to talk to SQL server from now on..

Ah, sounds pretty logic... 👍
Read something about readonly DB:s lately, was it those changes that made this an OPT-IN? 😉

Yup, that was part of the game :)
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Jonas Carlbaum<EMAIL_ADDRESS>Sent: Tuesday, May 5, 2020 10:46:09 PM
To: valdisiljuconoks/localization-provider-epi<EMAIL_ADDRESS>Cc: valdis iljuconoks<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [valdisiljuconoks/localization-provider-epi] Problem upgrading from 5.8.2 to 6.0.2 (#71)
Ah, sounds pretty logic... 👍
Read something about readonly DB:s lately, was it those changes that made this an OPT-IN? 😉
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvaldisiljuconoks%2Flocalization-provider-epi%2Fissues%2F71%23issuecomment-624268322&data=02|01||660a487c205c49a8e94808d7f12cf5f5|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637243047707473817&sdata=QQBDuVfho9pybxzdNGtT6vGJ6TXSc6ihrm7zzKabaFA%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAANPOC2NY5AG6JA4K3KDETLRQBUIDANCNFSM4MZQR4YA&data=02|01||660a487c205c49a8e94808d7f12cf5f5|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637243047707483812&sdata=8FPCScgmc4%2BVdeBHbsH97SyYAJ5TYh%2Bde%2BT4pbO0P98%3D&reserved=0.


 The 3 nodes are in the isolated network,and cannot access to the internet.
Command to set up the rancher is :
helm install ./rancher
--name rancher
--namespace cattle-system
--set hostname=rancher.ctyun.com
--set ingress.tls.source=secret
--set auditLog.level=3
--set auditLog.destination=hostPath
--set auditLog.hostPath=/opt/rancher/auditlogs/
--set auditLog.maxAge=20
--set auditLog.maxBackups=20
--set auditLog.maxSize=100
--set privateCA=true
--set rancherImage=rancher/rancher
The error logs show below:
2019/07/22 10:27:47 [DEBUG] REST UPDATE apis/management.cattle.io/v3//catalogs/system-library
2019/07/22 10:27:47 [ERROR] CatalogController system-library [catalog] failed with : Clone failed: Cloning into 'management-state/catalog-cache/2284e9d6c1e82c5a56a5cb7c1d5c0144835734b1b66da6f75122d99c7989618f'...
fatal: unable to access 'https://git.rancher.io/system-charts/': Could not resolve host: git.rancher.io
: exit status 128
2019/07/22 10:27:49 [INFO] Received SIGTERM, cancelling
2019/07/22 10:27:49 [FATAL] leaderelection lost for cattle-controllers

Up, still get this error

Air gap install is described at https://rancher.com/docs/rancher/v2.x/en/installation/air-gap/install-rancher/ and the system charts are bundled since v2.3.0. Lower versions have to follow https://rancher.com/docs/rancher/v2.x/en/installation/options/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0.
Let me know if this doesn't work.


Block launcher white screen

So, when I open the app there is a white screen for about 3 seconds and then the app closes and it doesn't show it was opened. I have an eclipsis, please help and fix.

I have downloaded the Pro version onto my hudl 1 running 4.2.2 and all I get is the white screen, however it does not go, just stays white, please help.


YouTube TOS and Appstore policy

Hey! I know this lib breaks youtube TOS, but can it be used for playing videos  in a appstore app without being rejected?

Yes, it can. See my reply in #114.

Thanks! But what if i decide to play a youtube stream using get_info link. Will then the app be rejected?

No. Again, please read issue #114.

Actually, our app was rejected because we are violating the YouTube TOS. So depending on the person reviewing your app, this library might not be approved.

According to App Store Review Guidelines History, section 8.6 was added on March 12, 2015 and states:

Apps that include the ability to download music or video content from third party sources (e.g. YouTube, SoundCloud, Vimeo, etc) without explicit authorization from those sources will be rejected

So yeah, this may have changed recently. :-(

I want to use this library in my iOS project but I am concerned about the Appstore Approval process. The youtube videos I will be playing  in my app will mostly have less than 5 minutes duration. So my question is

Can i use XCDYouTubeKit in my application if I want it to go live on app store?
Or should I go with the Official library provided by youtube.?

Quoted:
"According to App Store Review Guidelines History, section 8.6 was added on March 12, 2015 and states:
Apps that include the ability to download music or video content from third party sources (e.g. YouTube, SoundCloud, Vimeo, etc) without explicit authorization from those sources will be rejected
So yeah, this may have changed recently. :-(
"
But if just play(not download), it's ok right?

Yes, it's OK.

How do they even track if it is downloaded vs played? From the size of app storage?

They load the video, cut off internet connection, and if it's still working/playing, they consider it a violation of the guidelines.,

No, i believe that they have decompile tools and they can learn which libraries you have used easily. For example you are using nsurlconnection or nsurlsession for downloading. if you said i haven't let users to download videos they said why you've used nsurlconnection or nsurlsession so it is too hard to fool Apple.

That as well, but in our case we didn't even download the video / stored it for offline usage. It was still considered a violation.

Yes, unfortunately . i have an app thats using xcd. Apple is decomposing in review and always rejecting my app. Now it is impossible to use this framework anymore.

Can anyone else confirm that Apple is rejecting apps using this framework categorically as @niklaskorz and @codaman are suggesting?

Had an app approved 2 days ago without an issue...

i've tried 4 times and rejected. now, i sent a new binary 5 days ago without xcdyoutubekit , i'm waiting result

may be my app is in black list

@whoisdis do you allow users to download or only watch. how apple approve your app

Only watching in this app.

Okay, to make things clear: Our app has been accepted. But as I mentioned, we were rejected at first before we did not stop playback when the user loses internet connection, which Apple considered a way to make the content available offline. After "fixing" this, we had no further trouble using XCDYouTubeKit.

@niklaskorz May I know how to "fixing" that?

Hi, can anyone point me to a place in Youtube TOS that is being violated by the library? Or explain in more details what is violated?


Better error message needed when missing 'async'

When missing the async keyword on a function, the following error is displayed (Flutter, VSCode):
Unexpected text 'await'.
I think a better error will help beginners.

@danrubel this sounds like a parser issue.  Interested in having a look?


Backend

Added get for menu items.
supplies returns all
supplies/item/id/  reutrns that item by id

There's conflicts


Add --since <date> and --until <date> flags, and remove extraneous months from graph

These flags would filter down commands that list and graph activities.

This is now released in v0.29.


Perhaps Wordle should be a group project, and Product Filters should be done in Mix to make combative pairing easier.

Making this to document some thoughts and will circle back to them.

Product filters could instead be related to video games to make it part of the Games project, then product filters could be a mix project.

I'm not making Wordle a group project but will have students work in Pairs.
many students chose to use Mix for product filters, then paste their solution. This worked great.


ERROR: Failed building wheel for psycopg2 during install on ubuntu

ERROR: Failed building wheel for psycopg2  received when installed from latest master.
Workaround is to install psycopg2 package before install from conda forge conda install -c conda-forge psycopg2

closing the issue. if anyone has similar issue they could find it still.


convenience table or view with queryable transaction data

As currently the items and transactions tables store much of the useful data in a json object, might you be open to additionally exposing the data elements more directly as columns for query convenience?
It would seem that this could be done without data duplication using either views on the current tables, or alternatively by adding 'GENERATED ALWAYS' (https://www.sqlite.org/gencol.html) columns to the source tables. In either case, this would introduce a dependency on the JSON1 module json_extract function, to the extent there isn't one already.
Although you may have the SQL statements already (for your downstream use), I would be happy to write/send these if you were open to the above.
Thanks!

As long as the JSON is still available (since my workflow[1] depends on it), I'm open to changes like that - sounds like a useful improvement. I'd much rather have a single widely useful project than people needing to wade through a bunch of forks.
I'm not in a position to make changes to this project right now, but I'm happy to welcome any PRs that don't break existing (reasonable) workflows and seem generally useful.
The generated columns support is really cool. I don't know how I never encountered that in SQLite - i'm a huge proponent of SQLite in places other people would turn to a "real database" for querying purposes, but usually generate them from something else (like an Athena query) - but I can think of a lot of cases when this would have been useful. Thanks for the tip!
[1] I personally just use the data as a straight select * query in an importer for https://github.com/jbms/beancount-import and parse the JSON in Python, so I've never really cared that much about accessing the data from SQL queries - and when I do, I just use the json_extract method.

As long as the JSON is still available (since my workflow[1] depends on it), I'm open to changes like that - sounds like a useful improvement. I'd much rather have a single widely useful project than people needing to wade through a bunch of forks.
I'm not in a position to make changes to this project right now, but I'm happy to welcome any PRs that don't break existing (reasonable) workflows and seem generally useful.
The generated columns support is really cool. I don't know how I never encountered that in SQLite - i'm a huge proponent of SQLite in places other people would turn to a "real database" for querying purposes, but usually generate them from something else (like an Athena query) - but I can think of a lot of cases when this would have been useful. Thanks for the tip!
[1] I personally just use the data as a straight select * query in an importer for https://github.com/jbms/beancount-import and parse the JSON in Python, so I've never really cared that much about accessing the data from SQL queries - and when I do, I just use the json_extract method.

Sorry, a side topic, do you mind explaining how your workflow actually work? I am using beancount and beancount-import as well. Are you implementing the Source interface by providing the JSON? Thanks in advance!

@Xadoy Here's my custom source for beancount-import that reads the sqlite database this utility creates:
https://gist.github.com/mbafford/ff2ece412ce3188f9eab692b3823e917
Hacked together from other beancount-import sources as the first source I wrote. There's likely improvements galore to make in this, but I've basically left it untouched for years with no issues.


Add support for float and double precision data

Context: This PR will enable support for 64bit and 128bit complex datatypes, as well as restructuring the package class to allow for this extension.
Description of the Change: To enable the above, several modifications were required:

All gates and methods previously defined in Gates.hppand Gates.cpp are now part of the StateVector class
The apply functionality from Apply.hpp and Apply.cpp has been added to the StateVector class directly
The StateVector class was converted to a templated class, allowing all above functions to be compiled with the correct data types.
The C++ version was bumped from 11 to 17, to allow for use of newer features (inline static const, template argument deduction, constexpr if)
The StateVector class and methods can now be directly accessed from Python, using the StateVectorC64 and StateVectorC128 types. Use will require adaption of core Python implementation to utilise.
Tests based on the previous architecture format were removed. To be updated with upcoming Catch2 reformat.

Benefits: This enables use of data-types previously unavailable without casting (np.complex64), and will allow for memory and performance savings when used. Features spread over multiple additional classes and files are now made redundant, allowing for a more straight-forward implementation.
Possible Drawbacks: Structure of StateVector class has increased in size.
Related GitHub Issues:

Note: due to the required refactor of the testing infrastructure, the PR #115 should accompany this review, and will be merged in here before any merge with master.

[ch7526] [ch7552] [ch7067]

Really awesome work Lee. I left a few comments. Since I just joined, it might be due to my lack of familiarity, but I was wondering why there are gate definitions in two different places in the code, e.g. getCSWAP in StateVector.hpp and CSWAP in GateData.hpp, is this a necessity?

GateData.hpp was a carry-over from the googletest suite. I updated them, then decided to pull them out. You are completely correct that it can be pulled. I'll removed all the remnants of the old testing suite.

Recent changes have pulled the gate definitions from the class (fixes the CodeFactor complaint on StateVector), moved them to their own separate namespace, and used these to assist in the tests. This also fixed the second CodeFactor complaint.
I think anything else needed removal of unused libs can be moved to a separate PR and story once we have some spare time.

I will update the implementation for the new gate, then merge.


Composer packages

Does pib supports implementation of composer packages?

What do you mean by implementation of composer packages?
PIB is just a PHP interpreter running inside a browser. If you want to use compose packages, you can package or load dynamically the entire vendor folder into emscripten filesystem and include vendor/autoload.php as usual.

I assume by you saying including to emscripten filesystem is by using the flag --preload-file?

Yeah, --preload-file or load the file in JavaScript.

i have it loaded but after i included include('vendor/autoload.php'); in the code, the succeeding code below it doesn't seem to run.

What composer packages are you using?

@oraoto
Is it also possible to include files and directories (composer's vendor directory) dynamically after building? Or to dynamically build them?
What do you mean with "load the file in JavaScript"?
Also, is it possible to preload directory (instead of `--preload-file)?


Support react-native v0.33.0

Test react-native-storybook with RN v0.33.0 and extend the range on peer dependencies.

Do we still need to test this for every new release.
We don't do hacks now. So, it should be working under new releases automatically.
Isn't that the case.

Fixed with #71


Adding support for POST JSON payloads in IgluAdapter

RFC for allowing POST requests to the Iglu webhook endpoint using JSON

Very cool! Thanks @miike. Assigning to @jbeemster to review. @miike has signed the CLA.

I think this new commit hopefully addresses and fixes most of the comments made above.

Can we rebase this when ready to a single commit "Scala Common Enrich: add POST support to IgluAdapter (close #1184)"

Hey @miike, we are actually going to add some more functionality to this PR so I will be taking it over from here.  Thanks again for the PR!

@jbeemster No worries. Do you still want me to address the comments above or should I close this PR?

You can go ahead and close the PR.


copying styled text does not copy whitespace between spans

Describe the bug
look at the screen shot: the first paragraph was written and styled using the tool bar. Then, the whole text was selected, copied into the clipboard and pasted underneath
To Reproduce
Steps to reproduce the behavior:

enter some text and style parts of it
copy that text into the clipboard
paste it somewhere else

Expected behavior
The pasted text should look like the original
Screenshots

Desktop (please complete the following information):

OS: macOS Ventura
Browser Version 1.68.134 Chromium: 127.0.6533.88 (Offizieller Build) (arm64)

Nota bene: even worse: "cutting" the text into the clipboard also removes whitespace between spans - as a consequence, you can't move paragraphs
Amendment: I've experimented with the onCut, onCopy and onPaste callbacks. While I cannot see anything in the arguments of onCut and onCopy, the cleanDataprovided inonPasteclearly shows the "loss" of whitespace betweenelements... `s are kept, of course, but these badly affect selectabilty (sigh)

my problem seems to be old: there was a fix (https://github.com/JiHong88/suneditor/issues/189) which uses   instead of spaces - but that affects formatting and selectability

there seems to be a function in util.js which explicitly removes whitespace between some tags: htmlRemoveWhiteSpace

ok, here is my workaround - it's a horrible hack, but it works for me until the problem gets "officially" solved:
  my.SunEditor.onCopy = async (Event, ClipboardData, Core) => {
    let copiedHTML
      const ClipboardContents = await navigator.clipboard.read()
      for (const ClipboardItem of ClipboardContents) {
        if (ClipboardItem.types.includes('text/html')) {
          const ItemAsBlob = await ClipboardItem.getType('text/html')
          const ItemAsHTML = (await ItemAsBlob.text())
            .replaceAll('<span> </span>',' ')
          copiedHTML = Core.cleanHTML(ItemAsHTML)
          break
        }
      }
    if (copiedHTML != null) {
      const HTMLasBlob = new Blob([copiedHTML], { type:'text/html' })
      const HTMLItem   = new ClipboardItem({ 'text/html':HTMLasBlob })
      navigator.clipboard.write([HTMLItem])
    }
  }

The same applies to onCut

I have the same issue with the suneditor while pasting the text from vscode or word it removes spaces within the words and strangely joins them.

I encountered the exact same issue. Copying HTML from where-ever with tags (usually  in my case), the spaces before and after the tag are removed. I will be using @rozek's workaround as well, thanks a ton for the debugging work and providing a solution for others. :)


Android back button without scanning: FormatException (FormatException: Invalid envelope)

Exception has occurred.
FormatException (FormatException: Invalid envelope)

I not look Back Button

In the current dev release the scan() method returns a result object which contains a resulttype.cancelled instead of throwing an exception: https://pub.dev/packages/barcode_scan/versions/3.0.0-dev.1


Implement basic LUT filter

LUT color filter is basic thing for camera and video.

https://github.com/jsharp83/MetalCamera/pull/13


Got error message, about function of sanityinc/make-read-only, pp-display-expression in .el file in emacs 29.0.60

Hi @purcell ,
Could i ask the purpose of the function of pp-display-expression and sanityinc/make-read-only for what?
Because i  got error message when using emacs 29.0.60,
after eval function in .el files, always got error message in my computer.
(in scratch buffer or in org file buffer, the result of eval lisp function is as normal.)
after comment out functions in init.el,
finally, i found to comment out this will solved the situation:
 (advice-add 'pp-display-expression :after 'sanityinc/make-read-only)
in init-lisp.el file.
Even though that solved, it might be the problem of the version of emacs,
because i do not got such error in emacs 29.05
and i tested emacs 28.250, it also do not have such debug error message when i eval any function,
in the config files of https://github.com/purcell/emacs.d.git.
so i think if i want to report the issue, maybe need give some information for maintainer of emacs.
Here is the situation what i met and error message when i eval any funtion in .el buffer:
I was using emacs 29.05 , till three days ago.
duo to uding pacman only upgraded a single file, pacman broken,
so after reinstalled arch linux, then download emacs:
git clone https://github.com/emacs-mirror/emacs/tree/master
but master branch become emacs 30,
so i re-download old branch:
git clone -b emacs-29 https://github.com/emacs-mirror/emacs/tree/master
(by the way, today i found the branch 'emacs-29' disappeared suddenly.)
after make, it's version is emacs 29.0.60  (i did not remain src file of 29.05)
after test, emacs 28.250 do not have such issue,
emacs 29.06 and 30.050 has the issue, in my computer.
===error message start===
Debugger entered--Lisp error: (wrong-number-of-arguments ((t) (expression out-buffer-name) "Enable view-mode' in the output buffer - if any -..." (when (get-buffer out-buffer-name) (with-current-buffer out-buffer-name (view-mode 1)))) 3) sanityinc/make-read-only(sanityinc/make-read-only "*Pp Eval Output*" nil) apply(sanityinc/make-read-only (sanityinc/make-read-only "*Pp Eval Output*" nil)) pp-display-expression(sanityinc/make-read-only "*Pp Eval Output*" nil) pp-eval-expression((defalias 'sanityinc/make-read-only #'(lambda (expression out-buffer-name) "Enable view-mode' in the output buffer - if any -..." (when (get-buffer out-buffer-name) (with-current-buffer out-buffer-name (view-mode 1))))))
pp-eval-last-sexp(nil)
(if (and (mark) (use-region-p)) (eval-region (min (point) (mark)) (max (point) (mark))) (pp-eval-last-sexp prefix))
sanityinc/eval-last-sexp-or-region(nil)
funcall-interactively(sanityinc/eval-last-sexp-or-region nil)
call-interactively(sanityinc/eval-last-sexp-or-region nil nil)
command-execute(sanityinc/eval-last-sexp-or-region)
===error message end===

(by the way, today i found the branch 'emacs-29' disappeared suddenly.)

I know @tarsius fixed some mirroring config for that branch in the emacsmirror/emacs repo recently — note also that the contents of that branch were previously very outdated.
Probably that function takes an extra arg now — I should be able to add a fix in my config code to ensure backwards compatibility.

(by the way, today i found the branch 'emacs-29' disappeared suddenly.)

I know @tarsius fixed some mirroring config for that branch in the emacsmirror/emacs repo recently — note also that the contents of that branch were previously very outdated.

Good news, hope them will finish it smoothly.

Probably that function takes an extra arg now — I should be able to add a fix in my config code to ensure backwards compatibility.

That good news for me, because my English is not so well,
now i have one less issue for report.
and after test, today emacs 30.0.50 with your config have no error in eval, work fine.
Thanks your reply, and improve your config.
Your config is very good.
Have a nice day.

Thank you — I hope you have a good day too!


Add syntax highlighting to headers

It would be great to see syntax errors contextually in the headers section like this:

Instead of (somehow cryptic) error:

there is syntax highlighting, but this is invalid json!
json5 support will come with graphiql@3 and monaco


Free Disk Space for CI

Thanks for contributing!

What type of PR is this?

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #3277
Special notes for your reviewer:


[tool_crash] ProcessException: Failed to find "C:\src\flutter\bin\cache\dart-sdk\bin\pub.bat" in the search path. Command: C:\src\flutter\bin\cache\dart-sdk\bin\pub.bat, OS error code: 0

Command
flutter --no-color run --machine --track-widget-creation --device-id=emulator-5554 --start-paused --dart-define=flutter.inspector.structuredErrors=true lib\main.dart

Steps to Reproduce

...
...
...

Logs
ProcessException: Failed to find "C:\src\flutter\bin\cache\dart-sdk\bin\pub.bat" in the search path. Command: C:\src\flutter\bin\cache\dart-sdk\bin\pub.bat, OS error code: 0
#0      LocalProcessManager.start (package:process/src/interface/local_process_manager.dart:58:7)
#1      ErrorHandlingProcessManager.start.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:664:24)
#2      _run (package:flutter_tools/src/base/error_handling_io.dart:544:20)
#3      ErrorHandlingProcessManager.start (package:flutter_tools/src/base/error_handling_io.dart:663:12)
#4      _DefaultProcessUtils.start (package:flutter_tools/src/base/process.dart:425:28)
#5      _DefaultProcessUtils.stream (package:flutter_tools/src/base/process.dart:444:35)
#6      _DefaultPub.batch (package:flutter_tools/src/dart/pub.dart:283:34)
<asynchronous suspension>
#7      _DefaultPub.get (package:flutter_tools/src/dart/pub.dart:223:7)
<asynchronous suspension>
#8      FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:1170:7)
<asynchronous suspension>
#9      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1043:27)
<asynchronous suspension>
#10     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#11     CommandRunner.runCommand (package:args/command_runner.dart:196:13)
<asynchronous suspension>
#12     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:284:9)
<asynchronous suspension>
#13     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#14     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:232:5)
<asynchronous suspension>
#15     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:62:9)

[✓] Flutter (Channel stable, 2.2.1, on Microsoft Windows [Version 10.0.19042.1466], locale pt-BR)
    • Flutter version 2.2.1 at C:\src\flutter
    • Framework revision 02c026b03c (9 months ago), 2021-05-27 12:24:44 -0700
    • Engine revision 0fdb562ac8
    • Dart version 2.13.1

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\danilo\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: C:\Program Files\Java\jdk-11.0.10\bin\java
    • Java version Java(TM) SE Runtime Environment 18.9 (build 11.0.10+8-LTS-162)
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

[!] Android Studio (version 4.1.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to determine bundled Java version.
    • Try updating or re-installing Android Studio.

[✓] Connected device (3 available)
    • AOSP on IA Emulator (mobile) • emulator-5554 • android-x86    • Android 9 (API 28) (emulator)
    • Chrome (web)                 • chrome        • web-javascript • Google Chrome 98.0.4758.81
    • Edge (web)                   • edge          • web-javascript • Microsoft Edge 97.0.1072.62

! Doctor found issues in 2 categories.

Flutter Application Metadata
Type: app
Version: 1.0.0+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: 02c026b03cd31dd3f867e5faeb7e104cce174c5f
Plugins
path_provider_linux-2.1.0
path_provider_windows-2.0.3
shared_preferences-2.0.7
shared_preferences_linux-2.0.2
shared_preferences_macos-2.0.2
shared_preferences_web-2.0.2
shared_preferences_windows-2.0.2
sqflite-2.0.0+4

Please don't open multiple issues for same reason.
Closing this as self-duplicate of https://github.com/flutter/flutter/issues/97915


v2: automatically set content-type based on request body

Is your feature request related to a problem? Please describe.
node-fetch automatically sets the content-type header, based on the body given to new Response():
https://github.com/node-fetch/node-fetch/blob/6ee9d3186f8b4bdc4288ccf92f0a776aa17644ae/src/request.js#L72
It would be convenient to do the same in helix-fetch.

Request or Response?
helix-fetch does that already in the Request constructor: https://github.com/adobe/helix-fetch/blob/main/src/fetch/request.js#L96-L107

Request or Response?
helix-fetch does that already in the Request constructor: https://github.com/adobe/helix-fetch/blob/main/src/fetch/request.js#L96-L107


chore(master): release 1.4.12

:robot: I have created a release beep boop
1.4.12 (2023-05-17)
Miscellaneous

deps: update docker docker tag to v24 (871b587)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wayofdev/docker-build-deps/releases/tag/v1.4.12 :sunflower:


Integration Test - Release - 2022/01/26 07:23:05

Auto create for integration test. AnotherAuthorUpdatedArticleFlow

@prmerger-dev-2 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Docs Build status updates of commit 46f4ff8:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error: GitBranchDeletedOrForcePushed] Cannot sync git repo to specified commit because branch release-633e00a5e98c42f18c66e2bb6e8ea428 has been deleted or has been force pushed

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


windows: update libzmq dlls to v4 w/ sodium

This effort should be done from a PR to update these files:

x86 dll & lib: https://github.com/JustinTulloss/zeromq.node/tree/master/windows/lib/x86
x64 dll & lib: https://github.com/JustinTulloss/zeromq.node/tree/master/windows/lib/x64

If you have windows, here's the link to the msi so you can just install the DLL files into your program directory: http://zeromq.org/distro:microsoft-windows
Can someone help me with this? I'm not having at easy time at getting to this work right and I'm not sure why. When i replace the new DLLs, why doesn't the module load up anymore?
If someone with more windows experience could lend us a hand here I think everyone who uses this node module from windows will appreciate it

@kkoopa do you use windows?

Yes. I can take a look.

cool thanks :octocat:.

Weird indeed.

you get that silent failure where it just kills the node process?

Yes, I fixed some other points and get it to where it first tries to run a function from zmq, namely smq_version, where it terminates.

ditto, I found those DLLs made it so any operation from the binding kills node

It appears the handle is never saved or used for anything. That could be why things fail. The dll is supposed to be delayloaded, but I didn't see a delayload-hook anywhere. https://github.com/JustinTulloss/zeromq.node/blob/master/binding.cc#L1327

ah that's proabbly it, i missed that too, does it work with that line fixed?

just replacing the string there does not work, although the LoadLibrary call does succeed, but I don't see how further calls to zmq function will use this module handle.

Got it working.

wow! send the PR i want to see what u did

Done.


Remove redundant github danger/ prefix

Close: #1431
Technically all CI checks on GH from Danger having prefix danger/ what actually very inflexible if we can have an option to use danger_id CLI argument

By default check result is danger/danger what actually looks pretty strange. Probably just a danger  will be enough as a default configuration.

Without prefix CI check will be more flexible for some specific cases:
For example we having a monorepo with PlatformA and PlatformB. Both of them is independent and we having required checks:
PlatformA/check1
PlatformA/check2
PlatformA/danger
PlatformB/check1
PlatformB/check2
PlatformB/danger
Then we can create a some short circuiting bases on platform name kind of "If directory with PlarformA have no changes then set all checks PlatformA/* as Skipped"`

Or even if it required it can be declared --danger_id="danger/danger"

However those changes will break backward compatibility if previous danger check are required. And project maintainers should update required check name in this case.
What will be the best option in this case? IMO having 2 variables for declaring prefix and postfix is redundant and having only one looks straight forward for me

@orta

Nah, I don't think this is worth the major semver breaking change. I know I've relied on checking for danger/danger (which represents this repo, danger js has danger/danger-js) from other tools for example, which means there's very likely a bunch of people doing that now for danger/mysubset. It will not be obvious to them why something like this broke.
I wondered if including a / would be enough to allow skipping the prefix, but even that I can see would probably break someone's infra. I think this is going to need to be some other way, or just be OK with the prefix

No worries. Thanks for response.


Narrowing to non existing stream gives error message.

Solves #740

Automated message from Dropbox CLA bot
@krtkmj, it looks like you've already signed the Dropbox CLA. Thanks!

This looks pretty good, thanks @krtkmj!  Can you add a quick Casper test for this feature?

@timabbott added casper tests.

@krtkmj looks like this is failing some of the Travis CI checks; let me know if you need help understanding them!

@umairwaheed do you understand the check-templates issue this is getting?

@timabbott, for some reason Travis is not showing me the console output. I am getting this error. May be this is some temporary glitch, I'll check again in an hour.

@timabbott, this PR needs a rebase with upstream:master and instead of using trans we should use _().

This PR looks pretty good!  The only problem is that the narrow.js logic
This doesn't work correctly if you look at a stream that exists but the user has never been subscribed to -- stream_data only has data on streams that the user was subscribed to at some point in the past.  A reproducer is to create a new stream as one user in the Zulip development instance, switch to another, and try to narrow to the new stream -- you'll get the "stream does not exist" error.
Ultimately this is I think actually the same bug as we have open in #525 -- that we currently only have present in the Zulip frontend data about streams the user was at one time subscribed to.
So it seems like it's worth fixing that next.  Here's roughly the approach I'd take to make that work nicely:

Move the current should_list_all_streams logic into the backend and expose it to the frontend via page_params, so we can access it on the backend (we'll want it available in the backend for the following).  I'd add the backend code as a Realm method, e.g. realm.should_list_all_stream().
Modify home in zerver/views/init.py to send the data on public streams that the user is not subscribed to that are needed by populate_and_fill in static/js/subs.js as part of page_params, and remove the /json/streams call from subs.js to instead poll from populate_and_fill.  In this same PR, we may need to add code to update the list of public streams via the Zulip event system (this may involve some tweaks to apply_events and code to handle op=create and op=delete for the "stream" event types in static/js/server_events.js) -- the end goal here is that you should be able to have two browser windows open logged in as different users, both with the streams page open, and in one browser window we create a new stream, and the other should immediately rerender to show that new stream in the list.  And of course this behavior can be off if should_list_all_streams is disabled.
Then we can tweak this patch to query the now present in the frontend complete data for which streams exist
Finally we can add support for sending subscriber counts needed for #525 to the data flow.

Does that make sense @krtkmj?
(as a sidenote, @krtkmj please post a comment on PRs when you've addressed comments and are ready for another review -- GitHub doesn't notify me otherwise).

@timabbott In page_params, there is a list unsubbed_info and subbed_info. unsubbed_info doesn't contain any information of streams that user never subscribed. So, shall we add something like neversubbed_info or add never subscribed streams to unsubbed_info.

Yeah I think neversubbed_info is a reasonable name for that.  We may find we want to refactor it all to just one list with field(s) on the streams for what the user's status is, but I would start trying to write it as neversubbed_info; if it's reasonably clean, great; if it gets really messy, it's reasonable to try switching to another approach.

@timabbott

Added never subscribed streams info in neversubbed_info and exposed it to frontend using page_params
Never subscribed streams now show corresponding info.

@krtkmj it looks like this branch has lint errors (see the Travis CI output), can you fix those?

@krtkmj looks good overall! Should be pretty much ready to merge once you fix the lint errors.
One interesting case is if a stream is invite-only and the user is not subscribed. Currently this shows 'You are not subscribed to this stream. Want to join it?' but joining it isn't possible without an invite.
Now that we'll have this non-existing stream logic, I think we should either go with the 'Stream does not exist' message in this case, or remove the 'Want to join it?' link. @timabbott thoughts?

Yes, agreed; I think probably the "Stream does not exist" error message may be better approach of those.

@timabbott @neerajwahi I guess explicitly mentioning a message "This is invite only stream. And, you aren't invited" because after looking at a message like "Stream does not exist", user might end up trying to create a stream with same name.

@timabbott @neerajwahi Error message for invite only stream updated.

@krtkmj your test is likely failing because you're not handling the stream creation event in apply_events. You need to update state['never_subscribed'] so that the resulting state from calling fetch_initial_state_data + apply_events is identical to calling fetch_initial_state_data after the event.

@neerajwahi I updated apply_events to update state['never_subscribed'] whenever required. But, tests are still failing.
What I think is queries_captured() which allows a user to capture just the queries execucted. This might be possible that since I updated the code to get never_subscribed streams, number of queries must have reached 14( which in earlier case had be 4).

@neerajwahi One more thing that I noticed is that code for event['type'] == 'stream' and event['op'] == 'create' code is missing in apply_events() is not there. Am I missing something here?

@krtkmj The event['op'] == 'create' logic does not currently exist inside apply_events() since previously the creation of a new stream did not change the state. I believe you'll have to add that conditional branch

@krtkmj Added some inline comments to the diff.
Basically we can avoid having extra queries here by restructuring the code. We also need to add handlers for op='create' and op='delete' for the 'stream' event type in apply_events (this is why test_register_events is failing). Also, we should not include invite only streams in never_subscribed and instead treat them as if they don't exist -- we probably don't want to leak even the existence of those streams to someone who wasn't invited.

@neerajwahi Update handlers and optimized query to get never_subscribed streams.

@krtkmj Added inline comments. I don't think we need to add any extra queries for this feature

Thanks @neerajwahi. Earlier I thought filtering existing queryset does hit the database again. I was wrong(https://groups.google.com/forum/#!topic/django-users/U3pfYozg4wE). This is great knowledge.
Now, I can get all necessary information in only 4 queries except to get subscribers of never_subscribed streams I need to make a query. I am looking for a work around. Do you think there is another way without hitting back database for that?

@krtkmj You could modify subscriber_map to include all the stream IDs that you care about and avoid an extra query. Look at how the subscribed and unsubscribed lists are populated

@neerajwahi I can't figure out why this test is failing. Any suggestion?

Posted a comment that I think explains the cause of the test failure @krtkmj
@ddfisher FYI this is perhaps more incentive to fix the --fast-parser issues around wrong numbers of arguments and/or missing () around the argument list

Those are things I'd like to fix soon, but things are already significantly less confusing with the new, correctly-attributed error message.

@krtkmj Added inline comments

@krtkmj Thanks for the changes! Added some inline comments. Also your commit title has a typo ('subcibers' instead of 'subscribers')

@neerajwahi Updated this PR.

@krtkmj Will review your changes now
I just tested this in my VM and noticed an issue. Try the following and see if you can repro:

Log in as one account and create a public stream, then unsubscribe from that public stream
Log in as another account and narrow to the public stream you created

The error message offers to let you 'subscribe' to the stream (which doesn't seem to do anything)

@neerajwahi I followed steps you mentioned. One thing that I noticed any stream that has no subscribers is not present in stream_info (stream_data.js). And due to this, this stream is not listed in Subscription page(or streams page) for any user. It is possibly considered as inactive stream. And since stream is missing, this results in subs as undefined and is possibly creating problem.
@timabbott is there any reason behind this feature or is it a bug?

@krtkmj Another issue: subscribing to a 'never-subscribed' stream doesn't work.
To reproduce:

Create a public stream as one user
Log in as another user, narrow to the new public stream and try to subscribe

I tried this with both empty streams and streams with messages and got the following error in the JS console:
subs.js:816 Uncaught TypeError: Cannot read property 'subscribed' of undefined
If you look at that line in subs.js you'll see that it's checking sub.subscribed but sub is undefined for this never subscribed stream.

@krtkmj a stream that no longer has subscribers is treated as deleted, so that's why it isn't present in stream_info.  Is the problem you're thinking about what this page should do with a stream where everyone has been unsubscribed?  It's possible we need to do some tweaks to the model to make this page handle deleted streams correctly.
I'd like to get at least the backend piece of this merged soon.  @krtkmj can you rebase your branch to have at least two sets of commits:
(1) First, a series of commits that adds the neversubscribed_streams data on the backend, and has appropriate backend tests for this, but doesn't include the UI work for this feature (which is somewhat involved).  My hope is that we can get that finalized and merge it soon, and then you can finish #525 in parallel with finishing the UI work on this.
(2) Second, a series of commits that implements this feature.
Does that plan seem reasonable @neerajwahi?.

@timabbott That sounds reasonable to me
@krtkmj when you separate out the backend diff into its own commit could you please add some backend tests for never_subscribed_streams

@neerajwahi while I was trying to fix this, sub is undefined for a stream whenever stream either doesn't exist or user hasn't subscribed to it yet. Now, I am first checking if sub is undefined and stream is also present in neversubbed_info, then, subscribe to stream.
But, one problem that I faced is when a user A creates a stream when another user B is already logged in. Practically, for user B, page_params.never_subbedinfo should be updated without refreshing the page. If never_subbedinfo is not updated, narrowing through empty narrow feed subscribe button still fails. So, just before I start working how to update never_subbedinfo for all users, any suggestion how should I approach it?

@krtkmj Let's start by separating out the PR into a backend commit and one or more frontend commits as @timabbott suggests above.
Re: your issue above:
If you take a look at https://github.com/zulip/zulip/blob/master/static/js/subs.js#L684, you'll see that subbed_info and unsubbed_info are both used to populate the stream_data data structure and immediately deleted. If there's an update to a stream or subscription, the client is notified via an event and handles the update in this switch statement: https://github.com/zulip/zulip/blob/master/static/js/server_events.js#L56
Your neversubbed_info variable is a bit different since it doesn't correspond to a subscription in stream_data (and so you'll need to retain it in some fashion). You'll probably have to add a case to dispatch_event (in server_events.js above) to handle an event of type=stream and op=create / op=delete and update your neversubbed_info data structure with the newly created (or newly deleted) stream.

@neerajwahi Rebasing was pretty confusing me. So, I added never subscribed streams backend in another branch. Shall I make a pull request with RFC in header?

@krtkmj I'd prefer you rebase and keep the commits in this PR, partly to learn how to do it.
Check out this tutorial, specifically the interactive rebase (git rebase -i) section: https://www.atlassian.com/git/tutorials/rewriting-history/git-reflog

@neerajwahi Updated this PR.

@neerajwahi Updated this PR.

The new backend code is looking really good; I posted a few comments on potential cleanups

@timabbott working on it.

@neerajwahi it's up for review.

@timabbott Shall we merge never subscribed stream backend? Then, I can continue working on #525

I took a look, and merged the first commit (the simple variable rename).
However, the second commit has some bugs in how it handles the event updates.  I added two commits on  this branch as part of debugging this: https://github.com/timabbott/zulip/commits/neversubscribed
The first commit adds a test demonstrating the erroneous handling of a deactivated stream the user was not subscribed to, and additionally fixes the code so the test passes.  The second commit adds an additional test that this code causes to not pass, and has some partial work towards fixing the test (it'd be great if you can pick this up and fix the test from here; the issue is that no peer_add notice is being sent when another user subscribes to a neversubscribed_stream, and so the subscriber counts aren't actually being live-updated).

I'm not 100% sure we actually want to be keeping up-to-date the list of subscribers to unsubscribed and neversubscribed streams (it is potentially a waste of resources that will be problematic for very large teams; we can do some math to figure that out), so I'm not sure the second commit in my branch is a good direction.  It might be OK.

@timabbott I was working #d479519 and realised that in function do_add_subscription peer_add event is generated but yet tests are failing. Later, just trying somthing out, I replaced send_event(event, user_ids) with send_event(event, active_user_ids(user_profile.realm)) where user_ids is empty and later one is list of all users. test_subscribe_other_user_never_subscribed test passed. I could see peer_add event in apply_events. Thoughts?

I don't think I understand your question; can you post a link to the relevant code?

@timabbott, @neerajwahi : Updated this PR.

Looks good to me @krtkmj!
@timabbott thoughts? Do we want to add a gate around the new logic so that it doesn't apply for large teams?

Sorry for the delay looking at this!  I think the only potential performance concern is gather_subscriptions_helper itself (and in particular bulk_get_subscriber_ids); the event queue stuff is negligible because people change subscriptions way less than they send messages.  I suspect that might be more expensive than we want it to be for large teams, but I'm not sure what the size cutoff is for that (it might be basically fine).  I can plan to test that and report back on how urgent it is.
I think probably what we should plan to do for larger teams is just track the number of subscribers, rather than the list of subscribers (and then have the streams page query the list of subscribers only when it's needed); that should be inexpensive to handle (and cacheable!).  @krtkmj @neerajwahi can one of you open an issue for doing that as a follow-up?

I squashed the backend commits together to try to merge them here: https://github.com/timabbott/zulip/tree/neversubscribed, and noticed a few issues:

we're sending peer_add events for the users who are being subscribed themselves, which seems like it's going to be duplicate data (see 4a839b003a3d135760cf6c1e25bd6e174b8c4615 for a proposed fix)
I think we expand the tests to exercise the case of some users having already been subscribed and fully validate that everyone got the correct events here (in particular, I'd like the change I made in 4a839b003a3d135760cf6c1e25bd6e174b8c4615 to have hit a more interesting assert than just the count).

Otherwise that branch is ready to merge.  @krtkmj can you review my new commit and work on adding the needed test?

@timabbott: Cool, I will work on this and open an issue for the concern your raised above.

@timabbott: I didn't get a clear picture of what new test you want me add here. Can you please explain?

Sorry for the slow reply -- what I had in mind is a test similar to test_never_subscribed_streams but with some users starting out subscribed and validating that the events produced are correct.

I went ahead and merged the main backend commit for this with my tweak, since I think it's good.  You should still add such a test (and finish the frontend for this PR).  Thanks @krtkmj, I'm excited that we have the hard part of this finished!

@timabbott: I have added new test that you were discussing earlier to check whether users are getting correct events. See 0bb99a0.

https://github.com/krtkmj/zulip/commit/0bb99a0b27249f8d0a02114c43688429356e9526 seems to not be part of this branch?

@timabbott I updated this PR. For 0bb99a0 shall I create a new PR or add in this PR?

Just add it to this PR.

@timabbott Updated this PR.

@brockwhittaker I was wondering whether you'd take a look at this pull request and give a nonbinding code review?

Rebased this PR.

Hello @krtkmj, a Zulip maintainer reviewed this pull request over a week ago, but you haven't updated your pull request since. Please take a look at the requested changes and update your pull request accordingly.
Thank you for your valuable contributions to Zulip!

Hello @krtkmj, a Zulip maintainer reviewed this pull request over a week ago, but you haven't updated your pull request since. Please take a look at the requested changes and update your pull request accordingly.
Thank you for your valuable contributions to Zulip!

Closing this, since while it's a useful reference for doing this feature, there's enough merge conflicts that we'll probably want to type it in again before merging.
Thanks for all your work on this @krtkmj!


Clock Request: Daylight Map

How hard would it be to create something like the following?
https://s3-media1.fl.yelpcdn.com/bphoto/8hrn3DWyTbdI9DxT-MStNg/o.jpg

Hey. This was my first intention when I did the World Map Clock, actually I got there but I didn't like the final result.
https://github.com/jnthas/cw-cf-0x03/blob/main/worldclock.gif
I wrote the functions to do that in an old Display file that was in https://github.com/jnthas/clockwise/tree/main/firmware/lib/cw-gfx-engine. Have to seek in the git history.


d3.js mousemove

I am using d3.js library.
.on( "mousemove", $.throttle(1000, false, function() { console.log(d3.mouse(this) }) )
This returns only 1st result and some random good results afterwards. In general, it returns error:
[Error] TypeError: null is not an object (evaluating 't.sourceEvent')
b (d3.min.js, line 1)
mouse (d3.min.js, line 3)
(anonymous function) (index.html, line 74)
exec (jquery.ba-throttle-debounce.js, line 149)
If true is used for no_trailing it works but without the last event data (of course). I need false no_trailing.

how did you solve it?


Don't override svg fill

Don't apply a fill if there's already a fill in the attribute.

Resolves #165

Thank you, @davidmerrique. I’ll see if I can get out a new release right now.

@jonathantneal I appreciate it, thank you.

Published in v8.0.0. 😄

I use a class selector to set fill:
.Icon {
    fill: var(--fill-color, currentColor);
}

After updating sanitize.css to the latest version (8.0.0), the new svg:not([fill]) rule now overrides my own style because of its higher specificity.
Is this an issue?


To make a Todo file in markdown

Create a todo file which gives idea about the work which are still left, completed & on going.
It will help the Developers to the work more actively.
Note- The file should be in Markdown extensions.

Assign this to me


[iOS] Implement about DroidKaigi view

Issue

close #495

Overview (Required)

Implement about DroidKaigi view

Screenshot

Before
After

Can you merge from master for fxing CI? 🙏

@takahirom
Not working, did I do something wrong?

👀 🤔
https://github.com/ruby-git/ruby-git/blob/v1.5.0/lib/git/diff.rb#L152

@takahirom
It turns out there was a special character that cannot be notified on my Mac, and I have no idea why it exists.
Thank you for your support!🙏
The related PR is here.


[$500] Chat - Inconsistent behavior for different phone numbers with same country code is shown

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: <IP_ADDRESS>
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
Action Performed:

Launch app
Tap fab ---- Start chat
Search +84387025654 and select the contact
Note conversation page opened with header text hidden and no error is displayed
Navigate back to LHN
Tap fab --- start chat
Search +84387025656 and select the contact

Expected Result:
Inconsistent behaviour for different phone numbers with same country code must not be shown.
Actual Result:
Start chat with this number +84387025654, shown as hidden and no error displayed on opening conversation page.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/93399543/e861810b-6a9c-47f0-bbbb-8c17c8c07ad4
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~013a39fef35a8c97df
Upwork Job ID:<PHONE_NUMBER>413754368
Last Price Increase: 2024-01-20

It's the https://www.expensify.com/api?command=OpenReport service returning the error message. But the payload from the UI looks alright to be. I believe this is a service issue. Is there a transaction id Expensify returns so I can share the call for debugging on the back end?

Look like a backend issue.

@kbecciv actually, I don't think I can reproduce this? What specifically is the inconsistent behaviour? 🤔

I think this is similar: https://github.com/Expensify/App/issues/34839


NetVLAD++ Model RAM consumption?

Hi SoccerNet Dev Team,
I've managed to plug my own dataset into NetVLAD++, but am unable to train due to overloading my 32 GB of RAM.
I have ~80 matches of ~50 minutes of badminton matches with ResNet-152 features sampled at 5fps. After loading my dataset, I have ~18/32 GB of RAM used. The program gets killed while loading the model. I'm confused why, as it's only ~5.5 GB as shown in the TorchInfo summary below. I believe I should still have ~8 GB to spare. Is this a feature of NetVLAD++ specifically? I noticed that in #28 experiments were done with 60-90 GB of RAM.
Thank you for reading, and looking forwawrd to your insights!
TorchInfo Summary:
==========================================================================================
Layer (type:depth-idx)                   Output Shape              Param #
==========================================================================================
NetVLAD_plus_plus                        --                        --
├─Linear: 1-1                            [5236, 512]               1,049,088
├─NetVLAD: 1-2                           [44, 14336]               28,672
├─NetVLAD: 1-3                           [44, 14336]               28,672
├─Dropout: 1-4                           [44, 28672]               --
├─Linear: 1-5                            [44, 3]                   86,019
├─Sigmoid: 1-6                           [44, 3]                   --
==========================================================================================
Total params: 1,192,451
Trainable params: 1,192,451
Non-trainable params: 0
Total mult-adds (G): 5.50
==========================================================================================
Input size (MB): 42.89
Forward/backward pass size (MB): 31.54
Params size (MB): 4.77
Estimated Total Size (MB): 79.20
==========================================================================================

Hi @Wilann, NetVLAD++ considers a batch size of 256 by default (https://github.com/SilvioGiancola/SoccerNetv2-DevKit/blob/fb078911573409f48b85ee854dc8f09f432ee3a6/Task1-ActionSpotting/TemporallyAwarePooling/src/main.py#L133). Maybe you can try a smaller batch size.
If your issue come from the inference (or evaluation), that takes a full video as input, you might want to change that parameter (https://github.com/SilvioGiancola/SoccerNetv2-DevKit/blob/fb078911573409f48b85ee854dc8f09f432ee3a6/Task1-ActionSpotting/TemporallyAwarePooling/src/train.py#L229) that is the inference batch size (number of window inferred at once).

Hi @SilvioGiancola, thank you for your response! I tried with a batch size of 1 and the program crashes after loading the 300 training games. Would you happen to know what the minimum required RAM is to run the code?
Here's a screenshot:

Hi @Wilann , you are right, that implementation of NetVLAD++ actually pre-loads all the features in the RAM, for a faster training.
A quick work around for smaller RAM would be to train on a subset of your dataset. I see you are training on the 300 games from SoccerNet, you should be able to train on less videos. You said you only have 80 games in your dataset? Do you have he same issue when you train on less games?

Hi @SilvioGiancola, I actually managed to install more RAM into my PC, and NetVLAD++ is able to train on SoccerNet now (with alll 500 games, with ~55 GB of RAM used). For my own dataset I indeed have 80 games, and training works on that dataset as well (~20 GB of RAM used).
I also computed ResNet-152 features at 5 fps, and would like to train on those, but my GPU RAM gets overloaded. I tried lowering the BS = 1 as you suggeted above, but it seemed to make no difference. Do you have any suggestions on how I can optimize the code for 5 FPS?
Also, once the code reaches the dataloaders using the datasets using SoccerNetClipsTesting, it seems I get a spike in GPU RAM. I set-up my dataloaders something like this (as to try and replicate CALF's setup):
dataset_train = SoccerNetClips(...)
dataset_valid = SoccerNetClips(...)
dataset_valid_metric = SoccerNetClipsTesting(...)
dataset_test = SoccerNetClipsTesting(...)

But I noticed you used SoccerNetClips for dataset_valid_metric. By making this change, I also notice my GPU RAM spikes on my custom dataset, but not on SoccerNet. Do you have an idea why?
Thank you for your response - really appreciate it.
A visual on the spike with my changes made:

Hi @Wilann ,
Increasing the fps from 2 to 5 will incur +150% of memory usage. Your only solution would be to change the dataset.py.
In line 72, I am loading all the features in non-overlapping windows that I sample for training. That operation is memory intensive as it loads all the features in RAM. Your solution would be to push most of those operation in the getitem ([line 136] (https://github.com/SilvioGiancola/SoccerNetv2-DevKit/blob/fb078911573409f48b85ee854dc8f09f432ee3a6/Task1-ActionSpotting/TemporallyAwarePooling/src/dataset.py#L136)), but it will require further engineering for a decent loading speed.
Regarding the peak in evaluation, you can skip it all together and validate on the loss to prevent overfitting. Inhere, I was checking overfitting either with the loss or with the Spotting Average-mAP, but actually using the loss was good enough. For testing or inference, you will need to have a sliding window with a stride of 1 frame, or skip a few for a lighter inference. Again, it is loading all features in RAM for a sake of simplicity, but a better engineering could solve your issue.
I hope that will help you!
Cheers,

Hi @SilvioGiancola,
Thank you so much for your response!
I understand what you mean by moving the loading portion to the __getitem__ - will see what I can do about it.
About this part:

Regarding the peak in evaluation, you can skip it all together and validate on the loss to prevent overfitting. Inhere, I was checking overfitting either with the loss or with the Spotting Average-mAP, but actually using the loss was good enough.

I'm a bit confused when you say "validate on the loss". How do we validate on the loss? My understanding was that validation typically occurs on a portion of the dataset.
Also, you mention checking overfitting with the loss/Average-mAP - could you elaborate a bit more on how you did that?
Thanks so much for your time again - I really appreciate your help!

Hi @Wilann , Since the RAM peak appears in evaluation (testSpotting), you can skip that part. testSpotting requires a dense sliding window every second or so, hence will incur a lot of RAM usage as the whole video will be stored in overlapping windows. testSpotting is called twice, when "testing" and when "validating". For testing, it is obvious why, you want to evaluate your spotting performances on the test set. You can skip that testing for now. For validating (to avoid overfitting), you can stop the training when the validation loss start increasing. However, since the loss is a classification loss and not a spotting loss, you are not finding the best model for spotting but for classification. In order to be closer to the spotting task, you can then validate with the spotting performance (best epoch after spotting performance on validation drops) instead of the classification loss (best epoch after the validation loss increase). Note that it this choice is not much sensitive for the final performances, but validating on the loss instead of the spotting Avg-mAP will incur less RAM usage on your end.
That is also the reason why I have implemented 2 validation dataset, for classification or for spotting:
dataset_train = SoccerNetClips(...)
dataset_valid = SoccerNetClips(...)
dataset_valid_metric = SoccerNetClipsTesting(...)
dataset_test = SoccerNetClipsTesting(...)

Hi @SilvioGiancola,
When you say:

In order to be closer to the spotting task, you can then validate with the spotting performance (best epoch after spotting performance on validation drops)

Is the spotting performance the Average-mAP?
If so, how I understand it is that I basically I skip using the tesitng dataset for now, and only use testSpotting on the validation set. This change will save me quite a bit of RAM usage, but the downside is that I won't have another metric (Average-mAP from the test set) to gauge how well the model is generalizing - is this correct?
Another few questions:

Here in SoccerNetClips, what does this do? (I don't quite understand the comment)

2.1.  Here again in SoccerNetClips, what is being set to 0 here?
2.2. Here on the next line, why is 1 being added to the label index?

When sampling the dataloaders, for dataset_test, I'm getting something like:

Dataloader Length: 10
Batch Length: 3
Features Shape: torch.Size([1, 1866, 8, 2048])
Labels Shape: torch.Size([1, 1867, 2])
Index Shape: torch.Size([1])

Which I interpret as:
Dataloader Length: len(self.listGames)
Batch Length: 3
Features Shape: torch.Size([batch_size, 1866, window_size, 2048])
Labels Shape: torch.Size([batch_size, 1867, num_classes])
Index Shape: torch.Size([batch_size])

I believe 1866 and 1867 are being generated by this line, but what do they mean?
Thank you so much for your help again - really appreciate it.

Hi @Wilann ,
Yes, by spotting performance I meant the Average-mAP, and yes, you can skip the test set for sake of memory, and use either the SoccerNetClipsTesting or SoccerNetClips in validation, to check the overfitting on the spotting metric or on the classification metric, respectively.

That simply discard the events (annotations) that happen after the end of the game (1st and 2nd half). It is just a sanity check, the video features might not cover the complete video time due to their temporal receptive field and the temporal stride/padding. Here I am discarding the annotations at the end of each half game that does not contains any frame feature, hence from which I cant't learn from.
label_half1[frame//self.window_size_frame] is a one-hot encoding of the n+1 classes, including a background class with index 0. Line 86 initialize all one-hot encoding for all frame to be [1, 0, ..., 0] (background), so on Line 119 I change the one-hot encoding to reflect the correct class (not BG anymore but class label+1.
feat2clip Line 23 re-arrange the features along the videos into clips of specific length with a specific stride, in other words a sliding window of dimension clip_length extracted every stride indexes. It reduces the dimension from [video_length, feature_dimension] to [number_of_clips, clip_length, feature_dimension]. You might want to play with the padding parameter to have the same number of clips than the number of frame in you video/features (if you use a stride of 1), using "zeropad" to pad with zeros or "replicate_last" to replicate the last features and align with the number of frames in the video.
I hope that will help you.

Hi @SilvioGiancola,
Thank you for the reply again!
Following up on my questions:

I'm not sure what you mean by this:

the video features might not cover the complete video time due to their temporal receptive field and the temporal stride/padding

If I understand correctly, the extracted frame features may not span the entire video duration? What's the cause of this?
Also, printing something like this print(label_half1.shape[0], frame//self.window_size_frame) on my dataset for a single video gives me:
285 1
285 3
285 7
285 11
285 16
...
285 262
285 266
285 269
285 278
285 282

I can see that the sanity check works, and I believe label_half1.shape[0] is the number of frame features for this video, but am unsure how frame//self.window_size_frame results in the frame feature number.

What is a background class, and what is its purpose? I believe the predictions for this background class are being discarded in testSpotting line 251 & 252 - if this is true, then why include a background class at all?

Again, in line 119, why is the background class being set to 0 here? Why not just omit it all together if it's not being used? (I don't think its being used at least)

Actually, I just logged out all changes to idx in feat2clip, and realize the function is absolutely amazing - it really does work and the I/O shapes are exactly as you say! Regarding the padding, would you recommend either zeropad or replicate_last over the other, or would it depend on the task and dataset?

Thank you for your time again - really appreciate all the help.

In L119, I need frame//self.window_size_frame to be lower than the size of the tensor. It can come from the annotations or from a feature extraction with large strides that can't cover the end of the video. For instance, if you have 109 frames and a stride of 10 frames for your encoder, your last 9 frames won't be encoded, so if an action happens in the last 9 frames, you won't have any feature to recognize it. In case that occurs, L113 make sure it doesn't break by discarding the annotation.

Look at any spotting papers, we only want to localize when an action occurs. In most cases, no action happens in the video, hence it needs to be classified as BG. This is not a classification task "a goal or a foul" but a localization task. So I decided the BG is the class 0. See FasterRCNN for instance, it also classify an "empty" anchors as BG.

I don't have an answer, it's a design choice, check experimentally what's the best and stick with it. Google is your friend :)

Hi again,

I think I understand now. So if we pass frames at fps=2, for example, to the feature extractor it shouldn't be an issue, since there's no stride - is that correct?

Ah, I see. When there's no action it still needs to be given a class so the model can predict it, so it's just given class 0.

Thank you for the link! I actually remember reading about padding a while ago, but forgot why each type was used. Very helpful :)

An additional quesetion, since I'm labeling my own dataset on my own, its very time consuming and tedious. I decided to try and flip/mirror the videos and use the same labels to see if I could improve performance with minimal effort. In doing so, initially by including the labels from flipped videos improved performance, but it came to a point where the metric would instantly drop to 0. I found that the more labels from flipped videos I use, the earlier the metric would drop. Do you have any idea why this is the case?

Thank you for your time again - really appreciate the help!

It can come from the optimization, that diverged for some reason. Reduce the LR and check. Also, flipping the video inverts the past and future, so you might want to use the NetVLAD baseline instead of NetVLAD++/CALF.

Hi @SilvioGiancola, I actually managed to install more RAM into my PC, and NetVLAD++ is able to train on SoccerNet now (with alll 500 games, with ~55 GB of RAM used). For my own dataset I indeed have 80 games, and training works on that dataset as well (~20 GB of RAM used).
I also computed ResNet-152 features at 5 fps, and would like to train on those, but my GPU RAM gets overloaded. I tried lowering the BS = 1 as you suggeted above, but it seemed to make no difference. Do you have any suggestions on how I can optimize the code for 5 FPS?
Also, once the code reaches the dataloaders using the datasets using SoccerNetClipsTesting, it seems I get a spike in GPU RAM. I set-up my dataloaders something like this (as to try and replicate CALF's setup):
dataset_train = SoccerNetClips(...)
dataset_valid = SoccerNetClips(...)
dataset_valid_metric = SoccerNetClipsTesting(...)
dataset_test = SoccerNetClipsTesting(...)

But I noticed you used SoccerNetClips for dataset_valid_metric. By making this change, I also notice my GPU RAM spikes on my custom dataset, but not on SoccerNet. Do you have an idea why?
Thank you for your response - really appreciate it.
A visual on the spike with my changes made:

Any idea what is the recommend GPU RAM size for train and inference ? @SilvioGiancola

Hi @ldfandian , the message you quote contains your answer: ~55GB of RAM.

Hi @ldfandian , the message you quote contains your answer: ~55GB of RAM.

Thanks for the quick reply. @SilvioGiancola
And, I need to implement a getItem in order to reduce memory RAM. For GPU RAM, it looks to be tunable (by batch_size) to be as small as you like~  correct?

The answer is no. Please read the whole thread and you will understand why.


refactor(mission_planner/lanelet2_plugins): function updateRoute/clearRoute should be public override

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I'm convinced that this is not my fault but a bug.

Description
it's more like a refactor not a bug or even a question to confirm the current design.
In planning/mission_planner/include/mission_planner/mission_planner_plugin.hpp,
updateRoute() and clearRoute() are declared as public virtual:
https://github.com/autowarefoundation/autoware.universe/blob/main/planning/mission_planner/include/mission_planner/mission_planner_plugin.hpp#L44-L45
class PlannerPlugin
{
public:
  virtual void updateRoute(const LaneletRoute & route) = 0;
  virtual void clearRoute() = 0;
};

but in planning/mission_planner/src/lanelet2_plugins/default_planner.hpp, those 2 functions are declared as private:
https://github.com/autowarefoundation/autoware.universe/blob/main/planning/mission_planner/src/lanelet2_plugins/default_planner.hpp#L81-L82
class DefaultPlanner : public mission_planner::PlannerPlugin
{
public:

private:
  void updateRoute(const PlannerPlugin::LaneletRoute & route);
  void clearRoute();
};

I'd like to check if this is intentional or not.
Expected behavior
derived class should not hide public functions in base class
Actual behavior
in DefaultPlanner, those 2 functions are private, but if we use the base class's reference or pointer to call those 2 functions, they still can be called, which leads to confusion in code design.
Steps to reproduce
n/a
Versions
No response
Possible causes
make those 2 functions public override
Additional context
No response

@felixf4xu I don't think it was intentional. These functions should be public for mission_planner.
@kosuke55 (https://github.com/autowarefoundation/autoware.universe/pull/3334 reviewer) If there is a special reason, please let me know.


build(make): Refactor Makefiles for windows compatibility

overview
Replaces #598 and ~blocked by #804 (I'll switch the base branch of this PR to v3a once 804 is merged)~ unblocked!
This PR refactors the front-end Makefiles to make them completely cross platform compatible. api/Makefile is compatible enough for CI, so I didn't touch it, but it still needs work to actually be OK in Windows.
Also still TODO, we should add recommended OS setups to CONTRIBUTING. On my machine, I'm using chocolatey to install Git, Python v3.6, Node v8, and GNU Make:
choco install git.install --yes
choco install python --version=3.5.4 --yes
choco install nodejs.install --version=8.9.4 --yes
choco install yarn --yes
choco install make --yes

changelog

Added node_modules/.bin directly to PATH in Makefiles using yarn bin for cross-platform support
Replaced shell calls like rm and cp with shx
Refactored appveyor.yml to only use cmd and ps (no more Cygwin!) and run all project tests instead of just API and Run App
Switched datetime in Run App distributable filenames to UTC time
Aligned jest in all projects to v20

I would've used latest (v22), but I ran right into facebook/jest#4444

review requests
Need to make sure this works in all OSes!

I'd like to bump the python version to 3.6 on our dev systems and in CI (this matches what is running in the Pi)

@btmorr agreed that Python version should be aligned to 3.6, and I was also thinking we should have "Recommended Setups" in the contributing guide. Would you prefer that goes in this PR or a separate one?

It can be a separate one. I'm creating the Windows version as I test this PR


Corrupted dataset

Hi,
First of all, thanks for sharing this BarkNet 1.0 dataset : )
It seems a corruption problem in the zip dataset, for the linux system i got the error message:
End-of-central-directory signature not found.  Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive.  In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
I'm commited to work in a college project with yours data, it will be very useful it can be fixed as soon as possible
Thanks.

Check https://data.mendeley.com/datasets?query=Barknet
I have recently uploaded a dezipped version of the dataset on mendeley. Due to the size, it's broken in four parts.
Philippe Giguère
(envoyé par mobile/sent from phone)
On Sun, Sep 22, 2019 at 3:27 PM -0400, "AndersonCotrim"
<EMAIL_ADDRESS>wrote:
Hi,
First of all, thanks for sharing this BarkNet 1.0 dataset : )
It seems a corruption problem in the zip dataset, for the linux system i got the error message:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
I'm commited to work in a college project with yours data, it will be very useful it can be fixed as soon as possible
Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly,
view it on GitHub, or
mute the thread.
[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/ulaval-damas/tree-bark-classification/issues/3?email_source=notifications\u0026email_token=ABT6EJOOLBWU2ANVQNO45VLQK7BJBA5CNFSM4IZEEUPKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HM44XLA",
"url": "https://github.com/ulaval-damas/tree-bark-classification/issues/3?email_source=notifications\u0026email_token=ABT6EJOOLBWU2ANVQNO45VLQK7BJBA5CNFSM4IZEEUPKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HM44XLA",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]

I have rezipped the database here : https://storage.googleapis.com/barknet-1/BarkNet 1.0.zip
From: AndersonCotrim<EMAIL_ADDRESS>Sent: September 22, 2019 3:27 PM
To: ulaval-damas/tree-bark-classification<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [ulaval-damas/tree-bark-classification] Corrupted dataset (#3)
Hi,
First of all, thanks for sharing this BarkNet 1.0 dataset : )
It seems a corruption problem in the zip dataset, for the linux system i got the error message:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
I'm commited to work in a college project with yours data, it will be very useful it can be fixed as soon as possible
Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/ulaval-damas/tree-bark-classification/issues/3?email_source=notifications&email_token=ABT6EJOOLBWU2ANVQNO45VLQK7BJBA5CNFSM4IZEEUPKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HM44XLA, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABT6EJMJHFVOIGMHOYDVKI3QK7BJBANCNFSM4IZEEUPA.

Hi, thanks for replying. The crc problem was solved, but the new zip file only have 21 folders (classes), it is missing BOJ and BOP. Is it possible fix it?
Thanks.

Hopefully this time it’s good!
I restarted everything from scratch. Now the BOP and BOJ folders are in the .zip file.
https://storage.googleapis.com/barknet-1/BarkNet 1.0.zip
Best,
Philippe
From: AndersonCotrim<EMAIL_ADDRESS>Sent: September 22, 2019 3:27 PM
To: ulaval-damas/tree-bark-classification<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [ulaval-damas/tree-bark-classification] Corrupted dataset (#3)
Hi,
First of all, thanks for sharing this BarkNet 1.0 dataset : )
It seems a corruption problem in the zip dataset, for the linux system i got the error message:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
I'm commited to work in a college project with yours data, it will be very useful it can be fixed as soon as possible
Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/ulaval-damas/tree-bark-classification/issues/3?email_source=notifications&email_token=ABT6EJOOLBWU2ANVQNO45VLQK7BJBA5CNFSM4IZEEUPKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HM44XLA, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABT6EJMJHFVOIGMHOYDVKI3QK7BJBANCNFSM4IZEEUPA.

Hi again, thanks for helping. But now we have another problem, some files are corrupted in BOP and BOJ folder, there are few of them like 463_BOJ_56_GalaxyS5_20170607_154320_32.jpg (the training code doesn't work due this). Anyaway as there are few  of them, I have just delete them and now I can start the experiments. I think it will not be a big problem, just to report that there is this problem in dataset.

I have reuploaded a newer version, named it 1.001. I have manually checked now that the BOJ and BOP did not contain corrupt data. For some reason, I had corrupted versions on my desktop, and went back to the original disk.
Philippe
From: AndersonCotrim<EMAIL_ADDRESS>Sent: September 26, 2019 9:49 AM
To: ulaval-damas/tree-bark-classification<EMAIL_ADDRESS>Cc: Philippe Giguère<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [ulaval-damas/tree-bark-classification] Corrupted dataset (#3)
Hi again, thanks for helping. But now we have another problem, some files are corrupted in BOP and BOJ folder, there are few of them like 463_BOJ_56_GalaxyS5_20170607_154320_32.jpg (the training code doesn't work due this). Anyaway as there are few of them, I have just delete them and now I can start the experiments. I think it will not be a big problem, just to report that there is this problem in dataset.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ulaval-damas/tree-bark-classification/issues/3?email_source=notifications&email_token=ABT6EJNDEG424BG6IQVDOKTQLS4WBA5CNFSM4IZEEUPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7VT3WY#issuecomment-535510491, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABT6EJPP5JHHERIMY5YXM6LQLS4WBANCNFSM4IZEEUPA.


WebView Dark theme support

Task/Issue URL: https://app.asana.com/0/414730916066338/1174433894299360/f
Tech Design URL:
CC:
Description:
This PR introduces new flags on the webview configuration to automatically use website dark them if DDG browser theme is Dark.
Steps to test this PR:

Test:

Install the app
Visit SERP page
SERP should render in light mode
Change the app to use Dark theme
Go back to browser
Ensure SERP renders in dark mode
Go to SERP settings
Ensure dark theme is selected
Change the theme in SERP settings
Ensure selected theme is honored by webview
Now visit Twitter
Ensure it renders in dark mode
Change the app them to light
Ensure Twitter now is rendered in light theme

Internal references:
Software Engineering Expectations
Technical Design Template