Add no wait for consumer

This PR adds the no_wait option for consumers. It allows very fast reading without waiting for new messages.

If you're able to write a test as well that would be super, but no worries if not, I can add that after merge.

Yes, I will do it.

Update - the Go client has a separate method - FetchNoWait
But it also has this disclaimer:

Warning: Both Fetch() and FetchNoWait() have worse performance when used to continuously retrieve messages in comparison to Messages() or Consume() methods, as they do not perform any optimizations (pre-buffering) and new subscription is created for each execution.

I had a look in the nats go client and tried a little bit. They have a FetchNoWait function, which accepts a batch size.
Example: fetch, err := consumer.FetchNoWait(5)
If no message is available the messages chan closes immediately, which is similar to what this dotnet client is doing. They have a fetchResult, but its not public
What we could do in the dotnet client to adopt it like the go client would be the following (Its dummy code):
var fetchNoWaitOpts = new NatsJSFetchNoWaitOpts() {Serializer = new RawDataSerializer(), BatchCount = 5};
var pendingMessageCount = 0ul;
do
{
await using var sub = await consumer.FetchNoWait(fetchNoWaitOpts, cts.Token);
await foreach (var msg in sub.Msgs.ReadAllAsync(cts.Token))
{
await msg.AckAsync(cancellationToken: cts.Token);
Report(++count, stopwatch, $"data: {msg.Data}");
    pendingMessageCount = msg.
}

} while (pendingMessageCount > 0);

The example looks pretty much like ConsumeAllAsync() if it were to support a NoWait option

Can you maybe explain the difference between ConsumeAll and FetchAll? From what I understood from the go client the ConsumeAll returns all msgs, even if the MaxMsgs are reached. The FetchAll retrieves messages until the MaxMsgs are reached and then its done. Am I right?

My bad. ADR doesn't have this option (I thought it did)
https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-37.md#fetch-configuration-options

Can you maybe explain the difference between ConsumeAll and FetchAll? From what I understood from the go client the ConsumeAll returns all msgs, even if the MaxMsgs are reached. The FetchAll retrieves messages until the MaxMsgs are reached and then its done. Am I right?

Go client doesn't have *All methods. They are C# convenience methods (similar to ChannelReader.ReadAsync() and ChannelReader.ReadAllAsync()) which return an IAsyncEnumerable to be used in await foreach:
// This is same as...
await using var sub = await consumer.FetchAsync();
await foreach (var msg in sub.Msgs.ReadAllAsync())
{
}

// ...this
await foreach (var msg in consumer.FetchAllAsync())
{
}

// (same for consume)

You can think of Consume as never ending Fetch. Consume loop never ends until you stop it. The advantage is that Consume internally pre-fetches messages as they are 'consumed' while maintaining a healthy back pressure. So the client ends up getting uninterrupted flow of messages unlike Fetch where client needs to ask for more and wait (for a new fetch request to be processed) after every iteration.

If I understand correctly, the use case is:

You have a stream that is sealed, or not growing anymore (otherwise checking for checking if there are no messages would be unreliable, though still could be used).
You want to snap through all messages quickly and be notified where all messages are consumed.

My question is - Why not just just the Consume methods and just check the pending number on each message by calling Metadata() on each message?
That way client will do its bet to optimize fetching and handle most errors, while you have information about the state of the stream and consumer.

One use case of how we use nats is as EventSourcing database. With this we have to iterate over messages quite often. Even over streams which are still growing. The current consume and fetch always have a timeout which is to slow in our use case. With the noWait we can bypass this timeout and get instant feedback.
I will adapt the API in the same way as it is in the go nats client.

I think we need some form of 'end reason' for FetchNoWait() to detect if there are no messages and not fetch anymore. e.g. enum FetchNoWaitEndReason { Max, NotFound } or if we're going to expose underlying sub endReason we should expose it in other public interfaces too eventually.

I was also thinking about it. I would not expose it anymore (and I removed it right now 😀 ) to be consistent with the nats go client.
Maybe we can provide an example in the docs, to show the community of how to fetch all messages without the EndReason. Like my example I posted here.

@simonhoss but you don't get messages aftet fetch or consume finishes.
You get immediately when they are available.
There is no slowness introduced by the timeout.
The Timeout simply means that  Fetch was not able to deliver specified number of messages in specified time window because server did not have more.
Consume will also deliver messages as soonas they are avalilable, and if you want to check status of the stresm (i.e. how many messages are pending, delivered etc), use Metadata method on a message.
Maybe I'm not understanding something, but I see no reason here to loop fetch instesad of just calling Consume.

@simonhoss but you don't get messages aftet fetch or consume finishes.

Thats exactly what I want. I want to read the messages multiple times over and over again, also with some filters.
When I use the Consume I have to previous check if there are any message in the consumer, otherwise the consume would never return a message and wait forever, right?

Consumer is meant to be used like a standard NATS subscription: You call the Consume, and it will deliver you messges whenever they arrive. One consume can live thorugh the whole lifetime of your app.
I think I get what you need now - you want to know if there are messages in the stream or not.
I would love to understand what is the use case to know if there are messages beyond just getting them immediately when theyr're available and not getting them when there is nothing to get.
What is the difference between using consume with some timer equvalent to how often you were calling fetch and actually calling fetch no wait?
Sorry for all those questions - we're trying to understand the use case, as no wait can put the whole network and cluster under a big stress if it's called constantly over and over. If you scale that to many applications it can ramp up pretty badly.
Consume on the other hand optimizes both throughput and network/cluster utilization :)

As I wrote we're using nats as an EventSourcing Database in the context of Domain-Driven-Design.
In ddd you have to rebuild the aggregate every time when the system wants to save a new event. Rebuilding a aggregate works by replaying the subject and apply every event on the state, because the event must be validated against the state to verify if the event is valid and can be applied.
When adding an event we must guarantee that the new event is really the newest event and no other event was added in the subject during the replay and event persistence. For this we use the nats header Nats-Expected-Last-Subject-Sequence
A big requirement is, that the replay is as fast as possible, every delay in the replay must be avoided.
In that use case the NoWait helps a lot, but if you still thinking that Consume would still be a better idea I'm very interested to dig deeper into it.

@simonhoss thank you for the details, very helpful 💯
Please could you resolve the conflicts so we can merge.
PS We still have some questions and curious about the implementation details and were wondering if it can be done differently. A minimal demo project on GitHub (or somewhere) would be greatly appreciated if possible and when you can spare some time 😜 (just an idea, no worries at all if you can't)

I think it should be fine now.
I will add a sample repo. I'm also super interested in your opinion!
Thanks a lot guys


docs:  add snippet for a subcollection query

Add a beginner-level snippet that shows how to query a subcollection.
Similar PR for Android: https://github.com/firebase/snippets-android/pull/447
Note that I added two snippet tags: one for the convention used by the file and one for
the convention used on cloud.google.com
Risk Level

[ x ] No risk
[ ] Somewhat Risky
[ ] High risk

Pre-submit checklist

[ x] This PR follows the Google Developer Documentation Style Guidelines
[ x] This PR uses semantic line breaks

@thatfiredev -> Do you know why CI/CD might be breaking?

@nohe427 It looks like count() was introduced in FlutterFire Firestore v4.0.0, but CI is using 3.1.11:
lib/snippets/firestore.dart:827:28: Error: The method 'count' isn't defined for the class 'CollectionReference<Map<String, dynamic>>'.
 - 'CollectionReference' is from 'package:cloud_firestore/cloud_firestore.dart' ('../../../../../.pub-cache/hosted/pub.dev/cloud_firestore-3.1.11/lib/cloud_firestore.dart').
 - 'Map' is from 'dart:core'.
Try correcting the name to the name of an existing method, or defining a method named 'count'.

Done, but I encountered additional errors when trying to run the app locally. Filed bugs here:

#24
#25
#26

@jlara310 Thanks for filing those. Can you commit your current pubspec files after you ran that command so that CI can pick it up?

@jlara310 Thanks for filing those. Can you commit your current pubspec files after you ran that command so that CI can pick it up in this PR?

Yep. Done.

I am going to merge this since it seems the last few merges were also failing. I will then go through and try and fix the logged bugs. Thanks for making these changes and documenting the new errors. This will give us a great launching off point. @thatfiredev - I will ask you for reviews :)


Arkansas program admin seeing error upon login

Describe the bug
From Leslie: We’re having some issues setting up access for one of our Program Managers. Her email address has been added as a guest account in our AD system. We worked our way through the Microsoft authentication process, encountered some errors, deleted her account and re-added her in AD, but we still can’t seem to get her logged into the application. We’ve also cleared her History/cookies/cache in Chrome for “all time”
When she logged in, she saw the generic error message on the CiviForm page

I met with Leslie and Terry and we looked at the CiviForm logs and saw an error: Profile already contains an authority ID: Optional[iss: https://secure.login.gov..] - which is different from the new authority ID Optional[iss:https://login.microsoftonline.com/...]. Leslie confirmed the user had logged in first as an applicant then they tried to give her an admin account. Terry went into the database and found the account for the user and saw the old iss authority ID. He then updated it to the new ID: UPDATE accounts SET authority_id = 'iss:NEW' WHERE email = 'xxx';`

Updating documentation: https://github.com/civiform/docs/pull/479/files
Better error message: https://github.com/civiform/civiform/pull/8245

Went through another case of this with Leslie and Terry and this is now resolved.


Feature Request - Rate limiting based on time of day

Hi There,
Hope this is something that others would be interested in.
Looking for the ability to control data transfer rates based on time of day.
Best Regards
Rayno

Hi @Raynok  - you wrote me an email about this didnt' you which got lost in my inbox - sorry! Good choice making a ticket ;-)
What do you think about a command line interface a bit like this?

Hi Nick,
No worries, Alex and Klaus pointed me in the right direction.
I think, "Absolutely fantastic".
Going to give it a try and will let you. Great Project !!

Hi Nick,
Thank you for the pointer, there is a issue when issuing the command. Windows based computers.
My original command
rclone.exe copy Software:GLOBALSOFTWARE c:\support\ --transfers=1 -c --bwlimit=100
error: None
Testing command
rclone.exe copy Software:GLOBALSOFTWARE c:\support\ --transfers=1 -c --bwlimit=100 7-21,50"
Error: It is only necessary to use a unique prefix of the subcommand, eg 'mo'for 'move'.
Command copy needs 2 arguments maximum
If its something simple - I do apologize . I've tried use a .bat file with a set variable and got the same error.

That syntax -bwlimit=100 7-21,50 won't work at the moment, I just wondered what you thought about that syntax?
Namely having a 1 hour granularity and setting it like that.
If we think it is a good idea then we can implement something

The syntax makes sense and pretty simple to grasp though there must be some reference in the documentation ;-) . Setting the 1 hour granularity is perfect - I would do the same. You have my vote :+1:

+1 on this as well. That way while I'm sleeping, rclone can use up all my bandwidth without me having to mess with it.

I whipped up some systemd timers to do this, as well as to make sure folders are automatically backed up.
https://gist.github.com/AlexisEvo/3a803de79332b4e5fb83
It's gross, but it works. :)

Howzit - Still using rlcone and enjoying the new versions.
Alexis :+1: nice one . Didn't understand at first but did some reading. My environments are 99.9% Microsoft.
I've created a deployment package to distribute rclone to specific computers.
1.) The deployed package creates the folders and shares. Then...
2.) Downloads rclone + config file from google drive with a download rate I've set.
3.) Initiates the download of the files and logs it to a file.
4.) Once the download is complete it uploads the log file with the computer name and timestamp.
This gives me the ability to see the progress across multi pal wan sites and if there are no log files then its either busy downloading still or failed.
In a nut shell.
Control all the files on google and it mimics at the destinations. I can see the activity logs in google and this allows me to manage and view multi pal wan sites under one plane of glass (not perfect but best effort)
Thought I share how i try use rclone.

Not sure if it's helpful to anyone, but I'm using some rudimentary traffic shaping in Linux to help with bandwidth management while this native capability in rclone is being matured. Here's a link to some info: http://lartc.org/howto/lartc.ratelimit.single.html
Instead of the last line in the example that shapes based on destination IP I'm just shaping based on destination port (443) using this command:
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip dport 443 0xffff flowid 1:1
The Linux box I'm using for rclone sync commands is dedicated so I'm not too worried about shaping other port 443 traffic. It's hard to shape based on destination IP because most of these cloud services have a multitude of endpoints..

Why not use a multiple hh:mm,bandwidth type of configuration, having the currently selected bandwidth limits stay until superseded by the next configuration time slot? Something like:
-bwtimes="23:00,0 05:00,3000 08:00,500 20:00,3000"
This means:

At 5am, switch to 3000
At 8am, switch to 500
At 8pm, switch to 3000
At 11pm, switch to unlimited

Order should be irrelevant.
This format should be easy to parse and provides substantial flexibility with time selection.

@marcopaganini Does that mean you set --bwlimit as well as --bwtimes? I quite like there being only one command line, so as you suggested, but with the default bwlimit first, so something like --bwlmit ="500k 23:00,off 05:00,3M 08:00,500k 20:00,3M" (the SizeSuffix parser will parse the 3M or off for you).

That works well, but what is the purpose of the "default" bwlimit in this
case? If we have time slots, we'll always be covered by one of them at a
given time of the day.
On Sun, Oct 30, 2016 at 12:41 PM, Nick Craig-Wood<EMAIL_ADDRESS>wrote:

@marcopaganini https://github.com/marcopaganini Does that mean you set
--bwlimit as well as --bwtimes? I quite like there being only one command
line, so as you suggested, but with the default bwlimit first, so something
like --bwlmit ="500k 23:00,off 05:00,3M 08:00,500k 20:00,3M" (the
SizeSuffix parser will parse the 3M or off for you).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ncw/rclone/issues/221#issuecomment-257173633, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHFMBMNjvJ7mGThIDeo3X-OPc_vfh_NNks5q5PL_gaJpZM4Gi6Lh
.

@marcopaganini I guess the default limit is what you get until you get to one of the trigger times when the bandwidth limit changes. If you don't have that then you'll have to work out what the current limit should be at any given time.  You'll probably have to work that out anyway though - I imagine a go routine which runs every minute which resets the bwlimit to the limit appropriate to that time.
I think having two flags for setting the bwlimit may confuse people, but I'm happy to go with your suggestion for the exact syntax.

I totally agree with having one flag. I'm just a bit confused about the
"default" bwlimit. The way I was envisioning this was to always scan the
"time slots" and see where we currently fit, setting the bw limiter to that
value.
So, say, someone says 08:00,1M, 23:00,5M, it means that at any point in the
day, we'll be doing 1M (08:00 to 22:59) or 5M (23:00 to 07:59). Of course,
if only one time slot is used, then we'll be at that limit for the whole
day. :)
On Sun, Oct 30, 2016 at 1:01 PM, Nick Craig-Wood<EMAIL_ADDRESS>wrote:

@marcopaganini https://github.com/marcopaganini I guess the default
limit is what you get until you get to one of the trigger times when the
bandwidth limit changes. If you don't have that then you'll have to work
out what the current limit should be at any given time. You'll probably
have to work that out anyway though - I imagine a go routine which runs
every minute which resets the bwlimit to the limit appropriate to that time.
I think having two flags for setting the bwlimit may confuse people, but
I'm happy to go with your suggestion for the exact syntax.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ncw/rclone/issues/221#issuecomment-257175047, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHFMBMpz1OTbmujk8eY7Xm57wgrRbBhyks5q5Pd9gaJpZM4Gi6Lh
.

@marcopaganini I mis-understood your original plan - I thought it was an ordered list of instructions of time:bwlimit, but I see now that it is a timetable and yes I agree with you now :-)

Ooops, my bad! I should have made it more clear :) But yeah, the idea is
that it's a timetable. At any point, we're inside one of its "slots".
On Sun, Oct 30, 2016 at 1:18 PM, Nick Craig-Wood<EMAIL_ADDRESS>wrote:

@marcopaganini https://github.com/marcopaganini I mis-understood your
original plan - I thought it was an ordered list of instructions of
time:bwlimit, but I see now that it is a timetable and yes I agree with you
now :-)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ncw/rclone/issues/221#issuecomment-257176169, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHFMBOrj_LpqFXoOS17-pxvdFNP4mNW-ks5q5PuygaJpZM4Gi6Lh
.

@marcopaganini are you still working on this?

Hello Nick, haven't touched it yet (lack of time). I'll see if I find some
time to work on this during the holidays.
On Mon, Dec 12, 2016 at 3:14 AM, Nick Craig-Wood<EMAIL_ADDRESS>wrote:

@marcopaganini https://github.com/marcopaganini are you still working
on this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ncw/rclone/issues/221#issuecomment-266402713, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHFMBLdZwzJkXyoGP7U63LNA3eueH7T4ks5rHSx_gaJpZM4Gi6Lh
.

So, FYI I have a change coded, but it's still not working yet. PR coming
soon.
On Fri, Dec 16, 2016 at 1:28 PM, Marco Paganini<EMAIL_ADDRESS>wrote:

Hello Nick, haven't touched it yet (lack of time). I'll see if I find some
time to work on this during the holidays.
On Mon, Dec 12, 2016 at 3:14 AM, Nick Craig-Wood<EMAIL_ADDRESS>
wrote:

@marcopaganini https://github.com/marcopaganini are you still working
on this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ncw/rclone/issues/221#issuecomment-266402713, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHFMBLdZwzJkXyoGP7U63LNA3eueH7T4ks5rHSx_gaJpZM4Gi6Lh
.

Here is a beta with this new feature in: http://beta.rclone.org/v1.35-10-g3b0f944/ (uploaded in 15-30 mins)
Here are the new docs
--bwlimit=BANDWIDTH_SPEC
This option controls the bandwidth limit. Limits can be specified
in two ways: As a single limit, or as a timetable.
Single limits last for the duration of the session. To use a single limit,
specify the desired bandwidth in kBytes/s, or use a suffix b|k|M|G.  The
default is 0 which means to not limit bandwidth.
For example to limit bandwidth usage to 10 MBytes/s use --bwlimit 10M
It is also possible to specify a "timetable" of limits, which will cause
certain limits to be applied at certain times. To specify a timetable, format your
entries as "HH:MM,BANDWIDTH HH:MM,BANDWITH...".
An example of a typical timetable to avoid link saturation during daytime
working hours could be:
--bwlimit "08:00,512 12:00,10M 13:00,512 18:00,30M 23:00,off"
In this example, the transfer bandwidth will be set to 512kBytes/sec at 8am.
At noon, it will raise to 10Mbytes/s, and drop back to 512kBytes/sec at 1pm.
At 6pm, the bandwidth limit will be set to 30MBytes/s, and at 11pm it will be
completely disabled (full speed). Anything between 11pm and 8am will remain
unlimited.
Bandwidth limits only apply to the data transfer. The don't apply to the
bandwith of the directory listings etc.
Note that the units are Bytes/s not Bits/s.  Typically connections are
measured in Bits/s - to convert divide by 8.  For example let's say
you have a 10 Mbit/s connection and you wish rclone to use half of it

5 Mbit/s.  This is 5/8 = 0.625MByte/s so you would use a --bwlimit 0.625M parameter for rclone.

This is only in beta right?

@friendlytherapist it will be in the 1.36 release, currently in the beta, yes.

I'm having a bit of a problem with shell escaping.
I use a variable to store a bunch of my default switches but no matter how I escape the quotes, don't use them, or use different quotes, it isn't interpretting like i'd expect.
Example:
STD_SWITCHES="--stats 5m
--config='$RCLONECONF'
--transfers=16 --checkers=32
--one-file-system
--bwlimit="07:00,6M 23:00,10M" "
-># echo $STD_SWITCHES
--stats 5m --config='/home/robert/.rclone.conf' --transfers=16 --checkers=32 --one-file-system --bwlimit="07:00,6M 23:00,10M"
root@HS: ~
-># rclone $STD_SWITCHES   rmdirs
Error: invalid argument ""07:00,6M" for --bwlimit="07:00,6M: invalid time specification (hh:mm): ""07:00"
Usage:
...
...
2017/02/27 12:14:21 Fatal error: invalid argument ""07:00,6M" for --bwlimit="07:00,6M: invalid time specification (hh:mm): ""07:00"
I've tried all of the following:
STD_SWITCHES="--stats 5m
--config='$RCLONECONF'
--transfers=16 --checkers=32
--one-file-system
--bwlimit='07:00,6M 23:00,10M' "
STD_SWITCHES="--stats 5m
--config='$RCLONECONF'
--transfers=16 --checkers=32
--one-file-system
--bwlimit=07:00,6M 23:00,10M "
STD_SWITCHES="--stats 5m
--config='$RCLONECONF'
--transfers=16 --checkers=32
--one-file-system
--bwlimit=07:00,6M\ 23:00,10M "

I opened a new issue for it instead  Shell escaping #1183

Thank you for this feature, appreciate it.


WingetUI dark theme uses white titlebar

WingetUI dark theme uses white titlebar.

Hi @LoranRendel,
I'm afraid that the API that controls the color of the titlebar is only available on Windows 11, so this can't be changed on Windows 10 (at least as far as I know).
I'll check this again, but I don't think this can really be fixed

Hi @LoranRendel,
I'm afraid that the API that controls the color of the titlebar is only available on Windows 11, so this can't be changed on Windows 10 (at least as far as I know).
I'll check this again, but I don't think this can really be fixed

I use some other apps on Windows 10, which changes native titlebar to black. Maybe it also is possible for WingetUI.

I will investigate this

Fixed!


Document linker relaxation type.

List all linker relaxation type, and intentionally to not document zero-page relaxation for function call and pc-relative addressing instruction since we haven't reached a consensus yet (see #197 for more detail).
Fix #327

Prebuilt PDF: riscv-abi.pdf
Changes:

Rebase and grammar fix
Introduce  concept called relocation group to describe the we can't apply relaxation on only part of relocations which binded with same symbol.
Describe the R_RISCV_CALL or R_RISCV_CALL_PLT might also resolved as PLT stub of target symbol.
Describe R_RISCV_PCREL_LO12_[I|S] correctly in the relaxation condition, it will take symbol from the R_RISCV_PCREL_HI20

Ping, public review is closing soon, and we are trying to close all related PR within this week, let me know if you have comment or concern about this PR, we gonna to merge this PR is no further comment in next few days :)

Changes:

Grammar tweaks.
Address (most) @rui314's comments.

LGTM
FYI, we have implemented AUIPC → JAL/C.J/C.JAL and TP relaxation to mold, and it looks like they are working as expected.

Thanks everyone! finally we have more complete document for linker relaxation now :)


Custom message for URL field type?

I was wondering if there's an attribute for customizing the message for the field type? For example, on my "URL" field now, if someone enters an invalid URL, it will return "this value should be a valid url."
But what if i want to change that? Is that possible?

@Dayjo is correct.
Closing, but I'm taking note of @Dayjo's request.

Thanks @Dayjo. I apologize, but could you be a littler more specific on where I would that line of code to get the desired result?

@deronsizemore anywhere after parsley has loaded. Did you try it?

@marcandre Sorry, I did try it but for whatever reason I was under the impression it was supposed to go into the input tag in place of data-parsley-required-message="" parameter. I got it sorted out though after your response; never occurred to me to put it in script tag below where parsley.js is called. I did that and it works now. Sorry, just not real experienced with JS stuff. Thanks.

:+1:


Enable default sound

How to enable default sound using this library?

Oh, I found. It can be closed. apn_payload_set_sound(payload, "default");

x


Very slow graphics with newest chromeos version  90.0.4430.86, system basically unusable

Please paste the output of the following command here: sudo edit-chroot -all
name: xenial
encrypted: no
crouton: version 1-20200724165050~master:08dac970
release: xenial
architecture: amd64
xmethod: xorg
targets: audio,xfce
host: version 13816.55.0 (Official Build) stable-channel setzer
kernel: Linux localhost 4.19.176-11262-ge5fd762f9648 #1 SMP PREEMPT Mon Apr 19 20:11:30 PDT 2021 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Please describe your issue:
on an HP 11 G5, all of a sudden, scrolling and typing in browsers became unbearably slow, as did video.  video is very very jerky like  a set of still pictures taken every few seconds.  skype talking and other compute intensive things still work fine, its just graphical output that is bad.   upon reboot, chrome os was upgraded to 90.9.4430.86.
have seen before that things go bad like this and then on reboot there is an update, seems like chrome os does a partial update and messes things up before the full update?
but in those cases, updating the chroot with the current crouton has fixed things.  not this time.
it seems like somehow the X commands of linux are not getting passed to the correct libraries in chromeos, presumably they have hardware accelerated libraries for video and the like.
hope its something simple because this is friends main computer and its mostly unusable!
If known, describe the steps to reproduce the issue:
just boot chromebook, go into crouton with startxfce4, try to scroll or play video in firefox or chrome.   try to play downloaded video in smplayer with the same result.

I will put this as a separate comment so it might be noticed!
Looking around, I see in the Xorg log file the following

intel(0): SNA compiled: xserver-xorg-video-intel 2:2.99.917+git20160325
...
intel(0): Failed to submit rendering commands (Invalid argument), disabling acceleration.

The intel driver is old (but not for xenial) and has had problems, I think most current distributions no longer use it.   I still use a version on my desktop but it had trouble losing acceleration also, and I had to compile it myself with some recommended fixes that hadn't reached a distribution.   It seems like something similar could be the case here.   Perhaps I  can force the use of the standard modesetting driver in an Xorg config file.  Without any other advice forthcoming I will try that when I next get access to the device.


[FEAT] Dynamic filter feature

It would be great if the Pager allowed you to filter in place, the pager could have a property with a filter function on it, then when we modify the filter the pager would update in place.
An Example would be we:
We have 20 pages some with cards representing women some representing men.
A user then changes a settings view to say they only want to see men.
The filter is applied in place.
At the moment you could call the server and then get back your filtered data and reset the pager however I feel like this isn't nearly as good a solution as having the pager support filtering functionality.
At the moment if I try to performa filtering on the data after it's initial setup I get ContiguousArrayBuffer.swift:575: Fatal error: Index out of range. I suspect this is because the data behind the pager has been refreshed.
I could have a look at how to implement this myself if you point me in the right direction but I suspect this will be beyond my swiftui abilities :(
Thanks once again for creating the pager, it's a much needed project

problem is that the page isn't updated. You have items:
1  2  3  4  5  6

if you're on item 4 and filter even numbers, then there's a crash because you haven't updated the page accordingly. This related to #279 , I'll be making a quick fix there but ideally this would be handled by the client

Thanks, that makes sense will handle this issue on the client side


Create term io

TermIO is the updated and stream-oriented version of the library. It is much cleaner to use and better organized.


set GAME_VERSION to 3, no exit when only one player is dead

you already did the input piping, I only changed line 15 and 1259-1261

Thanks alot, didn't have the the time to finish it yet.

No problem. I wanted to test some things and when fixing the referee anyway, why not making a PR ;)
I am not 100% sure that it is correct, but the results look reasonable compared to arena submits.


FLINK-15975 Use LinkedHashMap for deterministic iterations

What is the purpose of the change
The test testMap in HiveGenericUDFTest invokes two methods HiveGenericUDFTest.init and HiveScalarFunction.eval.  These two methods depend on getConversion and toFlinkObject in class HiveInspectors. When the inspector is the instance of MapObjectInspector, it can return a HashMap. However, HashMap does not guarantee any specific order of entries. Thus, the test can fail due to a different iteration order.
In this PR, we propose to use a LinkedHashMap instead to guarantee the iteration order.
Brief change log
In flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/functions/hive/conversion/HiveInspectors.java, I use LinkedHashMap instead of HashMap in line 108, line 217, and line 316.
Verifying this change
This change is already covered by existing tests, such as (please describe tests).
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): ( no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (don't know)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Hi, which test has been failing?
There shouldn't any guarantee on the return of k-v in map object. Thus, instead of changing source code, we'd be better to change assertions in test code to cater to the indeterminacy, e.g. remove any assertion on ordering

The test testMap in HiveGenericUDFTest fails. As you suggested, we'd be better to change assertions in test code to cater to the indeterminacy.  So, I propose a new proposal.

can you rename the ticket and this PR?

A bit nicer version would be assertThat(Arrays.asList(result), containsInAnyOrder("1", "2", "3"))

Hi @testfixer, thanks for the fix.
one more thing: the title of PR and commit message should be of format "[FLINK-xxx][component] title" and they should be the same


zziplib: CVE-2018-17828

Motivation for this change
Fixes #61961.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@vcunat thank you so much for your review! I will know for the next time :)

Díky ;-)


ADD options

@SimenB would this be something you'd be looking for?
During serialization you can add options to choose which data types you want to ignore during serialization.

Released as v2.2.0

Awesome, thanks!


Fix flattening process to allow for additional field format options

Several issues like #31 #39  are caused by the flattening process which is slightly too simplified. According to the PDF specifications, the field format ("/Ff") is a 32 bit integer where only the last bit is relevant to make a pdf read-only. By setting the entire field to 1 as is done here we are losing out on quite a lot of functionality.
This patch fixes these issues by only setting the read-only bit to 1 and leaving all the other bits as they are.

@t-houssian Thanks for the great work, this is a nice package to work with.  Are you happy to merge this fix?

Looks great! Thanks @sachahu1


Tends to renew lock oddly/rapidly.

I'll be looking into this sometime in the near future. But hey, it works for now.


Added metadata for python page

Previous:
Title:
Python | New Relic Instant Observability
Description:
New Relic's instant observability quickstart boasts alerts, event loop diagnostics, and multiple customizable dashboards to detect and resolve issues before they impact end-users.
Now:
Title:
Python Application Performance Monitoring | New Relic
Description:
Python monitoring tool by New Relic helps business users and developers monitor Python application performance
Observation
I made changes on quickstart-metadata.json and then I have executed the code. But on http://localhost:8000/python/986df593-98db-485b-8e8c-dd866fa28813, I am not able to observe the updated title and description. Kindly guide me if I am going wrong somewhere.
Thank you. 🙂

Hey @pkosanam, I'm not sure why you aren't seeing the changes locally. I am able to see them in the preview!


Remove full state JSON serialisation of Services from InitialConfig

The configured service state should not be fully serialised into the json

merged into develop


DebuggerDisplayAttribute breaks in Rider

Describe the bug
When viewing the generated types in the debug window you get different messages depending on the type:
Int:
Underlying type: global::System.Int32, Value = Exception of type 'System.Security.VerificationException' was thrown: objRef is not object value

String:
Underlying type: System.String, Value = Failed to read field: Value does not fall within the expected range. The error code is E_INVALIDARG, orCOR_E_ARGUMENT, orWIN32_ERROR_INVALID_PARAMETER, or0x80070057.

What they seem to have in common is that the debugger breaks on stringifying the _value parameter.
It then doesn't even allow you to expand the objects as it doesn't get to the point of instantiating the proxy object.
If you remove the _value from
[global::System.Diagnostics.DebuggerDisplayAttribute("Underlying type: global::System.Int32, Value = { _value }")]

then you can expand the debug window. Unfortunately this then causes a stack overflow, because the DebugProxy has a reference to the ValueObject and it tries to recursively instantiate itself.
The solution is to take the important elements from the ValueObject in the DebugProxy and not keep a reference to the ValueObject in the DebugProxy.
I have no clue whether this is something that is contained to Rider's debugger or if this happens on other C# debuggers as well.
Steps to reproduce

Create a class

[ValueObject(typeof(string))]
public partial class TestThing
{

}

Create an instance

var test = TestThing.From("Test");

Do something with it

Console.WriteLine(test.Value);

Debug the line where you do something with it and look at the Variables section.

Expected behaviour
I see the Debug Proxy without any issues and don't get exceptions.

So definitely a Rider thing:
https://youtrack.jetbrains.com/issue/RIDER-79915
https://youtrack.jetbrains.com/issue/RIDER-79916
Probably not something you have to worry about as it should work. Not sure when it will get fixed however ...

Yeah I've seen some more bug reports on behavior that is slightly weird for these attributes in the bugtracker. I'm not sure how common they are in general. Your code is the first time I've seen them used, though they appear insanely useful in some situations.
So I do hope they fix those two bugs eventually, they've at least confirmed them. Maybe let's wait a bit see if its just solved there, if not it might still be an easy fix to add a configuration option that introduces a backing property instead of a field.

@Blackclaws can you work with Rider/Vogen without issues? In projects where I use Vogen I simply can't use Rider, as every time I touch a file where a VO is referenced, hundred of errors start to show (probably because Rider is re-creating the VO source files and had problems with that). I always have to use VS (a thing I don't like to do, but the other option is simply not usable).

That's not good. Perhaps I could #ifdef them out if you have something in your project properties. Something like NO_DEBUG_ATTRIBUTES?

@SteveDunn I don't think it is related to this attribute per-se. I asked @Blackclaws as it seems he's using Rider with Vogen.
I'm not sure what Rider does with source-generators, but before Vogen I used StronglyTypedId and had similar issues (but solved by simply closing/reopening the solution).
If I have to guess (based on my null knowledge of source generators), this seems to be related to how you hook into the pipeline.

@CheloXL I don't very often use Rider, but I just tried it on a project where I use Vogen, and couldn't see any issues. Have you got a small repro project you could share?

@Blackclaws can you work with Rider/Vogen without issues? In projects where I use Vogen I simply can't use Rider, as every time I touch a file where a VO is referenced, hundred of errors start to show (probably because Rider is re-creating the VO source files and had problems with that). I always have to use VS (a thing I don't like to do, but the other option is simply not usable).

Usually yes, it happens from time to time that Rider disables source generators after they threw an exception. Unloading and subsequently reloading the project in question fixes the issue. You can actually enable internal mode for Rider itself and view the project load log if you notice that this is something that always happens (https://rider-support.jetbrains.com/hc/en-us/articles/207327910 and then reload proejct and show logs).
Also make sure you're using the latest version, source generator functionality was improved a lot in the last major releases.

@SteveDunn Will try to create one. The project where I'm currently using it is quite big and of course I can't share it.
@Blackclaws Thanks, will try to see what happens. Usually on load there are no errors and everything is fine. Things start to break apart when I edit code that has references to VOs. Sometimes even a "move this class to another file" breaks everything.

👋 We are currently working to resolve this issue. You can follow the issue here. https://youtrack.jetbrains.com/issue/RIDER-79915/Rider-produces-an-exception-on-DebuggerDisplayAttribute-referencing-a-field

Thank you @khalidabuhakmeh - much appreciated. On a related note: If JetBrains want to use this repo to improve its handling of reporting errors from source-generated code, then I'd be happy to participate by making any necessary changes to how the source is generated. Both Rider and R# suffer from the same issues, e.g. when creating a new Value Object, they both display that Value is not a property of the type, even though it's source generated in the background.

I'm wondering if I should close this issue. Does the fact that having the debug attributes causes you to not be able to use Vogen? If so, I'll add something so that the can be turned off in Rider while its being fixed.

I'm wondering if I should close this issue. Does the fact that having the debug attributes causes you to not be able to use Vogen? If so, I'll add something so that the can be turned off in Rider while its being fixed.

It should be fine to close it, I opened the issue because I thought it was a Vogen issue, but since its a rider issue we can close it now.

I recently implemented an option that allows the user to omit the debug attributes. You won't get the useful functionality that VS shows, but at least it won't mess up the whole debugging experience in Rider.
You can achieve this by adding an assembly level attribute:
[assembly: VogenDefaults(omitDebugAttributes: true)]


shellenv: fix MANPATH and INFOPATH handling for [t]csh

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[ ] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

It turns out that control flow in csh is a pain. But we can rely on
test, which is thankfully POSIX.
Closes #17949.

Looks good thanks @carlocab!


Understanding of getPeerConnectionScript()

Does the same method (getPeerConnectionScript()) helps you in saving peerConnectionObjects in windows.pc variable while working with a conferencing system developed on Kurento SFU

hi @namvuCosmo
Can you please help here ?
Error is as follows:
This is a bug and must be fixed. The exception must be caught and thrown as KiteTestException
trace = org.openqa.selenium.JavascriptException: javascript error: APP is not defined

Hello,
This function doesn't save the peer connection object as windows.pc variable.
KITE is just using the existing variable for the peer connection object, exposed to browser console level, either it is windows.pc, or something else. This depends on how the developers name it.

This doesn't depend on the SFU either.
You can specify it in the config file, example for https://appr.tc:


Ensure frame is scheduled when root widget is attached

Description
After the change in https://github.com/flutter/flutter/pull/39535/ we were only scheduling the first frame on accident as a side-effect of the warm-up frame (SchedulerBinding. scheduleWarmUpFrame awaits SchedulerBinding.endOfFrame, which implicitly schedules a frame). This PR makes scheduling the frame more explicit right after the root widget is attached (that's the first point in time where the framework can produce a meaningful frame).
Thanks to @brandondiamond for figuring this out and bringing it to my attention.
Related Issues
#39494
Tests
I added the following tests:

Verify that attaching a root widget schedules a frame.

Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I signed the CLA.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (Please read Handling breaking changes). Replace this with a link to the e-mail where you asked for input on this proposed change.
[x] No, this is not a breaking change.


Removes install.php after ds build.

What's this PR do?

Removes install.php after ds build

Any background context you want to provide?
Recent Drupal core Critical PSA recommends removing install.php after installation: https://www.drupal.org/PSA-2015-001

:+1:


✨[feature Request]: adding a Wishlist.

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
We can add a feature like in which user can add products to Wishlist or to its liked product or something like buy it later.
Use Case
It will be helpful for the site to show recommendations and also for the end user to see the liked product without watching cart again and again.
Benefits
No response
Add ScreenShots
No response
Priority
Medium
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24EXTD contributor
[x] I have starred the repository

@Avansh2006 go ahead

@Anjaliavv51 Please add the level tag and also please check my PR #739...


README.md: mention the FastTract grant

Overview
Mention the FastTract NSF grant in the README.
Checklist
N/A

CI failure is a transient-looking docs issue (broken link in intersphinx stuff) — gonna ignore that and merge.


Can't use select widget to update interactive table

Software (anaconda in windows 10/11):
panel: 0.12.4
holoviews: 1.14.6
hvplot: 0.7.3
python: 3.9.7
I'm trying to use the select widget to choose which column appears in an interactive table. This works when returning a dataframe:
import panel as pn
from bokeh.sampledata.autompg import autompg
pn.extension()

s = pn.widgets.Select(options=list(autompg.columns))
def t(col1): return autompg[[col1,'cyl']]

pn.Column(s, pn.bind(t, col1=s))

But not for an interactive table:
import panel as pn, hvplot.pandas
from bokeh.sampledata.autompg import autompg
pn.extension()

s = pn.widgets.Select(options=list(autompg.columns))
autompg = autompg.interactive()
def t(col1): return autompg[[col1,'cyl']]

pn.Column(s, pn.bind(t, col1=s))

The reason I'm using the latter is because I also want to combine it linked selections as described here.

I can reproduce.


Docker: Backup solutions

Please use the following template when submitting issues:
Issue Template
Issue Type

[ ] Bug
[X] Enhancement

Description
When users run DefectDojo using Docker, they need some way of backing up their database. Currently, the DefectDojo image starts a database of its own, but also prepulates it in the build phase. That way, bind-mounting /var/lib/mysql to the host, which is the usual way of persisting important container storage, will crash the container (because of missing databases).
My idea would be:

Make the used database settings configurable through environment variables and default to SQLite (resulting in very few dependencies)
Move the default population of the database to the entrypoint script

That way, we could use the defaults in a dev environment and configure a, say, MySQL database connection in a second container or in a hosted environment for prod.
Another possibility would be to define and document a default workflow for backing up and restoring the DefectDojo data.

The original intent was only a dev/demo version of docker. Some in the community have done what you have suggested, here: https://github.com/kulinacs/django-DefectDojo/tree/docker and https://github.com/abelmokadem/docker-defect-dojo. Always happy to accept a pull request.

@aaronweaver Great. I will check kulinacs fork for that. @kulinacs, if you need help with that, give me a hint.


Private endpoints support

Provide support to use private endpoints for the services consumed by the Apps:

Databases (Postgress, MySql, SqlServer)
CosmosDb
Storage accounts
KeyVault
Application insights integration uses public endpoints
Redis Cache already implemented network integration.

Let's retake this feature once the refactoring is done.


Failing to build on latest dev

Was trying to build on the latest odin release but it comes up with a lot of errors, i've tried to go through and fix it myself to make a PR or something, but I don't know odin well enough.

Yeah that is not possible. The odin release is too old, but i have executables you can download: https://github.com/DanielGavin/ols/releases.
If you are on vscode it will automatically down them for you.

Ah I see, thanks.
On Sun, Aug 22, 2021, 10:06 AM DanielGavin @.***> wrote:

Yeah that is not possible. The odin release is too old, but i have
executables you can download: https://github.com/DanielGavin/ols/releases.
If you are on vscode it will automatically down them for you.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/DanielGavin/ols/issues/48#issuecomment-903274531, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ALCOGAKSYRJJZSH4OF77RQ3T6D75BANCNFSM5CSYIIUA
.


⚠ Secret-Cinema has degraded performance

In dc6f3b5, Secret-Cinema ($SC) experienced degraded performance:

HTTP code: 200
Response time: 1845 ms

Resolved: Secret-Cinema performance has improved in de41a84 after 7 minutes.


Force INFO loglevel for sentry/celery integration.

Coverage remained the same at 90.393% when pulling 16ac8656c509c20821eb956f45dd1d82445edbcc on johngian:fix-loglevel-sentry into 4b524add414d7ac083e2f0e81574722ed10761cc on mozilla:master.


adds stat and type efficacy changelog and some german translations

I am currently building a WinPhone App (just for fun and for just me) and I am using your data in an sqlite database. There is some data that I need so I added them to your database and I thought I'd share it.
Some stats where changed in 2nd and 6th generation. I added Pokémon stats for older generations and a new Stat "Special" which was SpecialAttack and SpecialDefense in first generation.
Also there were some type efficacy changes in 2nd gen.
I also provided some german translations.
Stat source:
http://bulbapedia.bulbagarden.net/wiki/List_of_Pokémon_by_base_stats_(Generation_I)
http://bulbapedia.bulbagarden.net/wiki/List_of_Pokémon_by_base_stats_(Generation_II-V)

Thanks for the pull request! It looks like you've combined several unrelated changes; it would be helpful if they were separate pull requests.

German translations. We love translations! I'd be happy to merge these, especially the prose translations. I'm curious what source you used for official game text.

Type efficacy changelog. I think the long-term plan is to add a generation column to type_efficacies, since we'd actually like to get rid of changelogs (see https://github.com/veekun/pokedex/issues/107 and https://github.com/veekun/pokedex/issues/58), but i suppose i could accept this as a stopgap.

Stat changelog.  Same story as the type changelogs. Also, we usually like to use better sources than Bulbapedia.
We never really decided how we wanted to handle the Gen I Special stat split (see https://github.com/veekun/pokedex/issues/8). The solution you've chosen is probably the least bad option. I don't think the Special stat should be marked is_battle_only, though.
I'm not sure how spline-pokedex will be affected by adding a new stat; it would probably need to be updated.

Mega evolution height. Looks good.

 Perhaps a bad merge?

Lastly, a general nitpick: some of the lines in the csv files are out of order; you should run pokedex load followed by pokedex dump to get the canonical order.

Hello and thanks for the reply and feedback.
You are right, it is better to split translations into another pull request. Let me answer your questions.

I will say something to every table.
ability_changelog_prose, ability_prose - The translations for abilities (there are more to come) I did for myself. Its somewhat difficult but I try to use the same phrases whenever possible.
item_category_prose, item_flag_prose, item_fling_effect_prose, item_pocket_names, move_battle_style_prose, move_meta_ailment_names, move_meta_category_prose - Same goes for them, I translated them myself
berry_firmness_names, characteristic_text, contest_type_names, pal_park_area_names, pokeathlon_stat_names, pokemon_habitat_names - I used pokewiki.de (the german bulbapedia) for them but I would love to rip them from a game - more to that later. They are in fact equal to the official names, at least for the actual generation. There were some changes to official text for characteristics over the games and even a contest category was beeing renamed in Gen 6.
I read about the generation/versiongroup columns, unfortunately after I submitted this request.
I am a while now about building my own Pokémon program, I started (and I am a little bit ashamed about that now) some years ago with building my own database by hand (meaning I would enter all data myself without programming help) with MS SQL Compact. Back then I had different tables for version groups like Pokemon_Gen1, Move_Gen1, Pokemon_Move_RBY and so on. This was a lot of duplicated data and it means you could not just get data by query but you must decide in the application which table to use.
The approach you take with version, versiongroup and generation columns is much better in my opinion, though it can be a little bit messy having to use version_group_id and generation in the same query.
You are totally right, the Special stat should not be marked is_battle_only, that was a mistake. Again I can only tell you what my older (and shaming) approach was. As I had Pokemon tables for each generation storing special in database was no problem at all, the pokemon_gen1 table simply had a Special column, rather than SpecialAttack and SpecialDefense. In my application however, I stored the Special stat in the SpecialAttack property, leaving SpecialDefense empty (nullable) which left me with nasty null checks every time I had to deal with it.
The approach I used here is far from perfect, cause of the Special Stat has it's own ID the join will somewhat not get the values from the changelog table. Maybe there is some elegant approach - I am far away from beeing a SQL expert - but I was missing it.
From my work I am used to rebase the master branch frequently into my work branch if it exists for a longer time, this is why I've done this here. I will close this request an open some new with the data separated and without commits from other requests.

As I hinted earlier I would love to rip official game text as well as Gen 2 locations but unfortunately I have no glue about how to do that. If you have some good references how to rip GBC roms I would love to extract some data into this database.


Revert "linux_*_hardened: use linux-hardened patch set"

Motivation for this change
This reverts commit 0d4f35efd4f905b322cd232e9740ce18f04270a0.
Trying to fix ofborg
hopefully I resolved the conflict correctly
example failure: https://gist.github.com/GrahamcOfBorg/77e6edd6891f7e7c84c4b4c3f2dcd51b
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

fixed with 6dfd563633ba75e814f821e3b431ebefdc69da51

@emilazy wants to work on a better script to avoid these fallouts.


Make FS0078 a warning instead of an error.

If you have #r "<something>" and <something> is invalid then the result is platform dependent and not consistent:
You either:

get a compilation error Error FS0078: Unable to find the file '<something>' in any of <list of locations>
or a warning warning FS0213: '<something>' is not a valid assembly name

depending on whether <something> is a valid file name, which itself is platform dependent.
Usually I'd suggest making this an error always.
However, considering FST-1027 or future extensions and https://github.com/fsharp/FAKE/issues/1780. I think it is reasonable to suggest to change FS0078 into a warning message.

Warning should be fine.  If the #r "assembly" is actually necessary, not finding it will yield a later error.  The one issue with making it a warning, is that I'm sure enterprising developers will write hunt and peck
resolution using it:
#r "path1\assembly.dll"
#r "path2\assembly.dll"
#r "path3\assembly.dll"
I would hate a developer to write the above ... but hey ... people will usually try the wrong thing and it's not as if fsi itself doesn't do exactly similar stuff internally.
Kevin

@KevinRansom Yes, I'm concerned by that too - people already do that mess with #I "...", e.g. look at this: https://github.com/BlueMountainCapital/FSharpRProvider/blob/master/src/RProvider/RProvider.fsx

@dsyme
yukk!!! but ...
when compiling, if msbuild fails to resolve a dll it doesn't fail the build.  The build only fails if the compiler tries and fails to find the type that should be found in the missing dll.
The warning really just allows for "consistency" with that.  Of course I held my nose when using the consistency argument, since generally I don't like it as an argument.  Often it seems to me that it is used as an argument to continue doing the wrong thing.
Kevin

@KevinRansom That file alone makes me desperate to have #r "paket: ... " and #r "nuget: ..." in the scripting model.  Paket's load script generation capability would also have helped

Of course I held my nose when using the consistency argument, since generally I don't like it as an argument.

Yes I agree. I only really suggested this because it would help with FAKE 5 until we have #r "paket: ..."...
There is also another inconsistency by the way: FSI and FSC will behave differently in these situations. FSC will yield warnings where FSI will emit errors. So it is inconsistent between platforms and between tools on the same platform.

While trying to figure out how to fix this I saw:

So a lot of warnings and error numbers are associated with the "simple" error scenario. And it is not obvious when the compiler emits what.

As now even @dsyme opened an issue over at FAKE (https://github.com/fsharp/FAKE/issues/2289). I'd really like this edge case to be sorted out so we can remove the recommendation to add // to every #r "paket: " string in FAKE.
Would a PR be accepted anytime soon for this?


Wallet detail transaction

Issue/Task Number: #947
Overview
Add wallet transactions tab.
Changes

Add wallet transactions tab.

Go to wallet details, should show a new design.
How can this change be deployed? Any special requirement?

Deploy as usual.

change base branch, cant review 6k lines of code :p

@nicholasmueller changed the branch.

@nicholasmueller fixed.


Error on requests that return no response body (202)

APIError: FetchError: invalid json response body  [...] reason: Unexpected end of JSON input
Per "Adding an asset" docs:

This returns a 202 empty response from our server

The following request is throwing an error when the library attempts to call .json() on the null response body
await imgix.request(`sources/${sourceId}/assets/add/${originPath}`, {
    method: 'POST',
    body: {},
  })

Hi @andrhamm, thank you so much for taking the time to report this issue. We truly appreciate your contribution to improving our project. 😊
We have acknowledged the bug you mentioned and our team will be looking into it. We understand how important it is to address such issues promptly, and we'll do our best to resolve it as soon as possible.
In the meantime, we would like to suggest an alternative approach to achieve the desired functionality. Instead of using the request method for the endpoint, we recommend utilizing the browser's Fetch AP or the node-fetch package to directly POST to the endpoint. Just note that you will need to set the correct headers.
Please keep in mind that this repository is currently experimental in nature, and as such, it may contain unexpected bugs and quirks.


feat: port AlgebraicTopology.SimplicialObject

[x] depends on: #3286
[x] depends on: #3340

This PR/issue depends on:

leanprover-community/mathlib4#3286
By Dependent Issues (🤖). Happy coding!

This PR/issue depends on:

leanprover-community/mathlib4#3286
By Dependent Issues (🤖). Happy coding!

This PR/issue depends on:

leanprover-community/mathlib4#3286
leanprover-community/mathlib4#3340
By Dependent Issues (🤖). Happy coding!

bors merge


How to get the duration and start time of each word?

ds2_model.decode_batch_beam_search

only return text and confidence

we can not get timing info now, we will add after.

is there problem solved ? i want the duration too.

is there problem solved ? i want the duration too.

please see examples/[datasets]/asr1 stage 4, CTC align has the function of forced alignment, which can get the corresponding time range of each word


Stop reload of live dev preview

Created preference in LiveDevMultiBrowser.js designed to disable live dev preview whenever a user imports files. Inside of src\filesystem\impls\filer\lib\FileImport.js I disable live dev preview before the import and then undo it after files are loaded.
This fix addresses the bug issue 2446 from the Thimble repo.
Link to the issue: https://github.com/mozilla/thimble.mozilla.org/issues/2446

@ericschv I think you know this, but in case not, you're failing lint:
/home/travis/build/mozilla/brackets/src/LiveDevelopment/LiveDevMultiBrowser.js
  978:40  error  'S' is not defined  no-undef
  978:41  error  Missing semicolon   semi
✖ 2 problems (2 errors, 0 warnings)

Can you fix those two issues and push a new commit?

@humphd yes I am aware. I will make the commit now.

@humphd I made the changes you requested

Thanks @ericschv, nice work!


Add typing for DefinePlugin

What kind of change does this PR introduce?
typing

Did you add tests for your changes?
no

Does this PR introduce a breaking change?
no

What needs to be documented once your changes are merged?
no

Waiting for Compiler types

@sokra @TheLarkInn this is good to go

@sokra this should be good to go


Reorganised algorithms.gradients

Fixes #9695 by reorganising algorithms.gradients into separate folders.
Summary
Reorganised algorithms.gradients as per issue #9695 and follows PR #9934.
Details and comments
Changed folder structure of algorithms.gradients, and collected files pertaining to base, finite_diff, lin_comb, param_shift, qgt, reverse, and spsa, into folders as per issue #9695. This also required that multiple files be refactored to reflect this change.

@a-matsuo The OP accidentally closed the original PR so I am tagging you in this new one.
@ica574 BTW it would have been possible to have re-opened the PR again, but this is ok given you have already done this replacement. And in general if you can avoid force pushing (I notice you did on the now closed PR) as it makes reviewing things harder since the delta changes get lost when you do that.

Can you also move ling_comb_qgt.py to the qgt folder? I want to put all qgt related files in that folder. Other than that, the PR looks good to me.

There is also the question of the documentation (this list of the files for Sphinx) at the top of the algorithms.gradients init files in terms of the organization there. This was discussed in the issue here https://github.com/Qiskit/qiskit-terra/issues/9695#issuecomment-1451260276 (as per below)

In the gradients in the docstring the organization/presentation to the user is not the same as the above structure (has Estimator grads, Sampler grads and QGT) - would you change that too, to reflect this organization or not. It would not affect the code in any way just how they are grouped in the docs.

I hadn't thought of that. I think it's better to group by gradient type in the docs, too, if the file structures are so.

Are you still of the same opinion @a-matsuo ?

@woodsp-ibm Thank you for mentioning it. I forgot about it.
@ica574 Can you group the documents in the __init__.py file by gradient types to align with the current structure?

The changes look good to me! @ica574 could you resolve the merge conflicts? Then I think we could merge this 🙂

I've made changes to fix the issues related to the broken references in the PR due to the new hierarchy in 'gradients'. Kindly review @a-matsuo @Cryoris .

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

67 of 67 (100.0%)  changed or added relevant lines in 22 files are covered.
10 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.007%) to 85.968%

Files with Coverage Reduction
New Missed Lines
%

crates/qasm2/src/expr.rs
1
93.76%

crates/qasm2/src/lex.rs
3
90.89%

crates/qasm2/src/parse.rs
6
97.58%

Totals

Change from base Build<PHONE_NUMBER>:
-0.007%

Covered Lines:
71480

Relevant Lines:
83147

💛  - Coveralls


Bug - Endpoint for /config/did/{id} does not exist

Since we can not query the /api/wallet directly there is no endpoint right now to request the did document via the issuer-api.
It would be great the query it either via GET /config/did/{id} or /config/did/doc/{id}.
The first solution would follow the typical pattern for REST APIs, but could lead to breaking systems. So the other option would be the besser way in case there will be more GET endpoints in the future.

fixed by https://github.com/walt-id/waltid-walletkit/pull/174


As an analyst, the goal is to acquire a detailed list of all capital cities within a specified region, organized in descending order based on population, so that I can use this information is crucial for understanding the population dynamics within the region, facilitating regional planning, and aiding policymakers in making informed decisions related to infrastructure, economic development, and public services.

Finished issue.


JsResourceHandler doesn't use fallbacklanguage

Hi,
In our episerverframework config we have a setup like this:
<localization fallbackBehavior="Echo, MissingMessage, FallbackCulture" fallbackCulture="en-GB">
    <providers>
      <add name="db" type="DbLocalizationProvider.EPiServer.DatabaseLocalizationProvider, DbLocalizationProvider.EPiServer" />
    </providers>
  </localization>

and in out initilizer I have added this code (just to test if it works, but it doesn't change the outpout
ConfigurationContext.Setup(ctx =>
{
    ctx.DefaultResourceCulture = new CultureInfo("en-GB");
});

And the epi language settings, these is the settings:

In epi localization mode I can see these translations

When I look at the translation for the key "Norrona.Commerce.Framework.Translations.Profile.BetaTesterLabel" in english in the jsl10n, then I see it fine (because it is filled out)
But when I go to the norwegian jsl10n, then it is not in the resource at all
Live example urls here:
English: https://www.norrona.com/jsl10n/Norrona.Commerce.Framework.Translations.Profile?lang=en-GB
Norwegian:  https://www.norrona.com/jsl10n/Norrona.Commerce.Framework.Translations.Profile?lang=nb-NO
This is for the latests packages (5.7.1)
if I add this code instead
ConfigurationContext.Setup(ctx =>
{
    ctx.DefaultResourceCulture = CultureInfo.InvariantCulture;
    ctx.EnableInvariantCultureFallback = true;
});

 UiConfigurationContext.Setup(ctx => { ctx.ShowInvariantCulture = true; });

Then it kinda works, I get the translation in english, but the InvariantCulture is not possible to edit, if it was possible to edit then that would have been a solution for us

thanks for detailed description! have you ever got provider working ?? :troll:

lucky enough yes ;) Just not the JsResourceHandler
So, will you have a fix to tomorrow? :troll:

Thanks for starting working on this right away.
I found a workaround for now that works for us (still need to confirm the fix with my boss)
The fix is to do this:
@Html.GetTranslations(typeof(Translations), "en-GB")
@Html.GetTranslations(typeof(Translations))
It is still wrong as we have setup that norwegian should be the fallback language for sweden, but at least we don't get an "undefined" error anymore, and actual translation. and in english it gets loaded twice, but still better then no translation :D
Thanks for the help, have a good day

however, there is a catch. this workaround is sort of working - will overwrite (deep merge), but.. JsClientResource will (and have to) follow the same fallback rules that is configured for built-in Episerver localization service (<localization> element). Language specific fallbacks are effective only for content and not for resources (XML file based ones for example). So it still will not be possible to configure fallback for different languages to use different fallback languages.
This is planned feature in v6.0 -> to eliminate this Episerver localization service fallback language limitation and give possibility for developers to configure whatever settings make sense for resources.

Would you still want to try out proper fix (preview of v5.7.4) or you are good with this workaround? I will be able to send you a preview after couple hours..

and also - in order to get correct behavior of the fallbacks - you need to change order of behaviors to:
<localization fallbackBehavior="FallbackCulture, Echo, MissingMessage" fallbackCulture="en-GB">

Echo and MissingMessage -> you have to check with your requirements how exactly missing resource translations should behave in your site..

Yes, a proper fix is the best and I will be happy to test it out :)

Tried it out now, and it doesn't looke like it works, but thanks anyway. Looking forward to v6.0!
Thanks for your great effort

I do have a working sample with 5.7.4 maybe we have some specific difference?

Maybe, I may have done something wrong with the preview packages you sent me too. I will just wait for the offical packages and try again then :)

v6.0 might not come so quickly..

I ment I will wait for the 5.7.4 package in the official nuget reporsitory as I may have done something wrong when I installed it localy :)
I understand that v6.0 will take time and not be here, but for now we at least have a backup solution even if 5.7.4 doesn't work

if 5.7.4 will not solve your issue, I'll retire and will spend all days picking mushrooms in forest..

Thanks, I will test during the week, right now I'm testing if
<localization fallbackBehavior="FallbackCulture, Echo, MissingMessage" fallbackCulture="en-GB">
makes the edit mode slow... because suddenly the whole epi edit mode is slow, and I think I have backtracked it to that change
It may have something to do with issue #31 though

might be. please run SQL profiler to check details under the hood. might be cache misses all the time..

It only happens in production, lovaly it is fine... so any idea how i can check in production? Never used sql profiler 😂

DXC?

On-prem as it is much cheaper, so i have access to the servers

Okey, Learned how to run SQL profiler I think :')
I see a lot of there
exec sp_executesql N'SELECT      [Project2].[Id] AS [Id],      [Project2].[ResourceKey] AS [ResourceKey],      [Project2].[ModificationDate] AS [ModificationDate],      [Project2].[Author] AS [Author],      [Project2].[FromCode] AS [FromCode],      [Project2].[IsModified] AS [IsModified],      [Project2].[IsHidden] AS [IsHidden],      [Project2].[C1] AS [C1],      [Project2].[Id1] AS [Id1],      [Project2].[ResourceId] AS [ResourceId],      [Project2].[Language] AS [Language],      [Project2].[Value] AS [Value],      [Project2].[LocalizationResource_Id] AS [LocalizationResource_Id]     FROM ( SELECT          [Limit1].[Id] AS [Id],          [Limit1].[ResourceKey] AS [ResourceKey],          [Limit1].[ModificationDate] AS [ModificationDate],          [Limit1].[Author] AS [Author],          [Limit1].[FromCode] AS [FromCode],          [Limit1].[IsModified] AS [IsModified],          [Limit1].[IsHidden] AS [IsHidden],          [Extent2].[Id] AS [Id1],          [Extent2].[ResourceId] AS [ResourceId],          [Extent2].[Language] AS [Language],          [Extent2].[Value] AS [Value],          [Extent2].[LocalizationResource_Id] AS [LocalizationResource_Id],          CASE WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C1]         FROM   (SELECT TOP (1)              [Extent1].[Id] AS [Id],              [Extent1].[ResourceKey] AS [ResourceKey],              [Extent1].[ModificationDate] AS [ModificationDate],              [Extent1].[Author] AS [Author],              [Extent1].[FromCode] AS [FromCode],              [Extent1].[IsModified] AS [IsModified],              [Extent1].[IsHidden] AS [IsHidden]             FROM [dbo].[LocalizationResources] AS [Extent1]             WHERE [Extent1].[ResourceKey] = @p__linq__0 ) AS [Limit1]         LEFT OUTER JOIN [dbo].[LocalizationResourceTranslations] AS [Extent2] ON [Limit1].[Id] = [Extent2].[ResourceId]     )  AS [Project2]     ORDER BY [Project2].[Id] ASC, [Project2].[C1] ASC',N'@p__linq__0 varchar(8000)',@p__linq__0='/contenttypes/ProductRowBlock/properties/LastWeek/help'
where the last part /contenttypes/ProductRowBlock/properties/LastWeek/help usually is diffrent
Not sure if that will help at all though. Looks like the query goes quick, but guess if it is many it may have some performance issues?

yeah, this seems like CMS is building resource cache at startup.. hmmm


Converted mediapipe palm detection coreml model cannot be deployed on IOS app

Issue Type
Bug
OS
Mac OS
OS architecture
x86_64
Programming Language
Python
Framework
CoreML
Download URL for tflite file
https://github.com/google/mediapipe/blob/master/mediapipe/modules/palm_detection/palm_detection_full.tflite
Convert Script
docker run -it --rm \
  -v `pwd`:/home/user/workdir \
  ghcr.io/pinto0309/tflite2tensorflow:latest

tflite2tensorflow \
  --model_path palm_detection_full.tflite \
  --flatc_path ../flatc \
  --schema_path ../schema.fbs \
  --output_pb

tflite2tensorflow \
  --model_path palm_detection_full.tflite \
  --flatc_path ../flatc \
  --schema_path ../schema.fbs \
  --output_no_quant_float32_tflite \
  --output_dynamic_range_quant_tflite \
  --output_weight_quant_tflite \
  --output_float16_quant_tflite \
  --output_integer_quant_tflite \
  --string_formulas_for_normalization 'data / 255.0' \
  --output_tfjs \
  --output_coreml \
  --output_tftrt_float32 \
  --output_tftrt_float16 \
  --output_onnx \
  --onnx_opset 13 \
  --output_openvino_and_myriad

Description
when I deployed the converted mp palm detection coreml model on IOS app, I got error like:
2022-02-15 16:31:50.577312+0000 MLModelCamera[12602:4637828] [espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Generic error": [Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1

seems like the resizeBilinear op causes this problem. Is there any way to fix this issue? Thanks in advance.
The related code in tflite2tensorlfow.py is:
        elif op_type == 'RESIZE_BILINEAR':
            input_tensor = tensors[op['inputs'][0]]
            size_detail = interpreter._get_tensor_details(op['inputs'][1])
            size = interpreter.get_tensor(size_detail['index'])
            size_height = size[0]
            size_width  = size[1]

            options = op['builtin_options']
            align_corners = options['align_corners']
            half_pixel_centers = options['half_pixel_centers']

            def upsampling2d_bilinear(x, size_height, size_width, align_corners, half_pixel_centers):
                if optimizing_for_edgetpu_flg:
                    return tf.image.resize_bilinear(x, (size_height, size_width))
                else:
                    if optimizing_for_openvino_and_myriad:
                        if half_pixel_centers:
                            return tf.image.resize_bilinear(
                                x,
                                (size_height, size_width),
                                align_corners=False,
                                half_pixel_centers=half_pixel_centers
                            )
                        else:
                            return tf.image.resize_bilinear(
                                x,
                                (size_height, size_width),
                                align_corners=True,
                                half_pixel_centers=half_pixel_centers
                            )
                    else:
                        return tfv2.image.resize(
                            x,
                            [size_height, size_width],
                            method='bilinear'
                        )

            output_detail = interpreter._get_tensor_details(op['outputs'][0])
            lambda_name = get_op_name(output_detail['name']) + '_lambda'
            output_tensor_lambda = tf.keras.layers.Lambda(
                upsampling2d_bilinear,
                arguments={'size_height': size_height,
                            'size_width': size_width,
                            'align_corners': align_corners,
                            'half_pixel_centers': half_pixel_centers},
                name=lambda_name
            )(input_tensor)
            output_tensor = tf.identity(
                output_tensor_lambda,
                name=get_op_name(output_detail['name'])
            )
            tensors[output_detail['index']] = output_tensor

Relevant Log Output
2022-02-15 16:31:50.577312+0000 MLModelCamera[12602:4637828] [espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Generic error": [Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1

Source code for simple inference testing code
No response

Sad to say, I don't own an IOS environment, so I can't test it. I'm not entirely sure if it will work, but could you please check if the CoreML model you downloaded from the following URL works?
model_coreml_float32.mlmodel.zip
docker run -it --rm \
-v `pwd`:/home/user/workdir \
ghcr.io/pinto0309/tflite2tensorflow:latest

wget https://github.com/google/mediapipe/raw/master/mediapipe/modules/palm_detection/palm_detection_full.tflite

tflite2tensorflow \
--model_path palm_detection_full.tflite \
--flatc_path ../flatc \
--schema_path ../schema.fbs \
--output_pb

tflite2tensorflow \
--model_path palm_detection_full.tflite \
--flatc_path ../flatc \
--schema_path ../schema.fbs \
--output_coreml

Thanks for the reply. Just test the model on the IOS, still get the same error.
2022-02-16 12:04:35.315685+0000 MLModelCamera[12740:4737111] [espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Generic error": [Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1
2022-02-16 12:04:35.315728+0000 MLModelCamera[12740:4737111] [coreml] Error computing NN outputs -1
2022-02-16 12:04:35.315755+0000 MLModelCamera[12740:4737111] [coreml] Failure in -executePlan:error:.
failed to perform

I checked the layer 240 of the graph, pretty sure the ResizeBilinear caused this problem, as it has also been mentioned in many onnx->coreml conversion, e.g. :
(https://github.com/onnx/onnx-coreml/issues/474)
(https://github.com/apple/coremltools/issues/1105)

I don't think the number 240 in the error message is directly related to the error. The error occurs in the operation named model_1/model/add_24/add.
[Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1

The shape of the tensor shown in the error message is obviously wrong.
Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1]

This is because there are no operations in this model that process in 5 dimensions.

I also noticed that the new height and new width in resizebilinear layer are all 0. Is that something related?

No, not at all. It is correct that new_height and new_width are set to zero, as per the specification. Since the second input is set to [24, 24], ResizeBilinear (upsample) will produce an output of size 24 in height and width.

The way I see it, it's a bug in CoreML.

Thanks for the reply. I also think it's a bug in coremltool. However, I try
to find a way to work around it. My initial thinking is to use upsample2d
to replace image.resize method in ResizeBilinear Op. Currently, got the
model deployed on iOS, but the size of the final output is incorrect. I
will keep tracking and let u know once I sort it out.
Katsuya Hyodo @.***> 于 2022年2月16日周三 下午2:34写道：

I don't think the number 240 in the error message is directly related to
the error. The error occurs in the operation named
model_1/model/add_24/add.
[Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1
[image: image]
https://user-images.githubusercontent.com/33194443/154284738-9a42f361-3645-4248-983c-df8bad275b36.png
The shape of the tensor shown in the error message is obviously wrong.
Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1]
This is because there are no operations in this model that process in 5
dimensions.
I also noticed that the new height and new width in resizebilinear layer
are all 0. Is that something related?
No, not at all. It is correct that new_height and new_width are set
to zero, as per the specification. Since the second input is set to [24,
24], ResizeBilinear (upsample) will produce an output of size 24 in
height and width.
[image: image]
https://user-images.githubusercontent.com/33194443/154285717-5a510b64-e6f6-4bf0-8fcd-240956d6df76.png
The way I see it, it's a bug in CoreML.
—
Reply to this email directly, view it on GitHub
https://github.com/PINTO0309/tflite2tensorflow/issues/27#issuecomment-1041554494,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AV7EQ3SUWDMYQY3PDEUKVZLU3OYV7ANCNFSM5ORKIUNQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi, Katsuya
I am glad to tell you I find a way to work around. I was inspired by this article https://machinethink.net/blog/coreml-upsampling/
Instead of using tf.image.resize, I use tf.compat.v1.image.resize_bilinear. The hacked code I used for coreml specifically is as follows:
        elif op_type == 'RESIZE_BILINEAR':
            input_tensor = tensors[op['inputs'][0]]
            size_detail = interpreter._get_tensor_details(op['inputs'][1])
            size = interpreter.get_tensor(size_detail['index'])
            size_height = size[0]
            size_width = size[1]

            options = op['builtin_options']
            align_corners = options['align_corners']
            half_pixel_centers = options['half_pixel_centers']

            def upsampling2d_bilinear(x, size_height, size_width, align_corners, half_pixel_centers):
                if optimizing_for_edgetpu_flg:
                    return tf.image.resize_bilinear(x, (size_height, size_width))
                else:
                    if optimizing_for_openvino_and_myriad:
                        if half_pixel_centers:
                            return tf.image.resize_bilinear(
                                x,
                                (size_height, size_width),
                                align_corners=False,
                                half_pixel_centers=half_pixel_centers
                            )
                        else:
                            return tf.image.resize_bilinear(
                                x,
                                (size_height, size_width),
                                align_corners=True,
                                half_pixel_centers=half_pixel_centers
                            )
                    else:
                        y = tf.compat.v1.image.resize_bilinear(x, [size_height, size_width], align_corners=True) # slightly better way, which works on IOS
                        # y = tfv2.keras.layers.UpSampling2D()(x) #   no matching solution in COREML
                        # y = tfv2.image.resize(                  #   no matching solution in COREML
                        #     x,
                        #     [size_height, size_width],
                        #     method='bilinear'
                        # )
                        return y

I see. I will add an option to enable CoreML optimization. Thank you for doing some meaningful research! :smile_cat:

Commit: ab7d9625aa2abc9d006e03d27f6d7f534e83f4db
Release: https://github.com/PINTO0309/tflite2tensorflow/releases/tag/v1.19.0


Solver RK4 still surprising behavior

I had trouble with the homemade rk4, and decided to verify if our solvers were working well. In consequence, I coded a very simple couple ODE (dampened harmonic oscillator) and compared results with its analytical solution.
The Homemade RK4 still have a weird behavior. Here is a difference between the numeric and theoric value :

The slope is 1/2, it should be 4, this system surprisingly behave as a system of order 1/2.

It seems to me that the code is right, so I do not know why it is not working https://en.wikipedia.org/wiki/Runge–Kutta_methods
def _rk4(dydt_func=None, dt=None, y=None, t=None):
    """
    a traditional RK4 scheme, with:
        - y = array of all variables (all ode)
        - dt = fixed time step
    """
    dy1_on_dt = dydt_func(t, y)
    dy2_on_dt = dydt_func(t + dt/2., y + dy1_on_dt * dt/2.)
    dy3_on_dt = dydt_func(t + dt/2., y + dy2_on_dt * dt/2.)
    dy4_on_dt = dydt_func(t + dt, y + dy3_on_dt*dt)
    return (dy1_on_dt + 2*dy2_on_dt + 2*dy3_on_dt + dy4_on_dt) * dt/6.

Will try to investigate this in a few weeks (mid-December probably) if the problem is still there by then :-)

Yes ! I'll focus on using the rk8 from scipy for the moment :)

I coded a Lorenz system to check the  qualitative system, for the moment both the rk2-scipy and the homemade rk4 are wrong :)

I went a bit further on the subject :

an Rk1 has been added for the very-simple case
Coded a convergence test (see testConvergence_DampOsc and both Rk1/Rk4 are not power-law which imply that either the convergence test is wrong or that the problem is somewhere else
testConvergence_DampOsc is showing that the first slope estimation is wrong, it might give insight on the problem

The way I see it : The initial condition on the first derivative is not respected, even if well noted in get_summary.

Maybe there is an error in the order of resolution ?

So I had a look using the DampOscillator with Perfect preset (because it is easy to see any problem) and it appears to me that the homemade rk4 behaves correctly if the time step is fine enough.
Below I compare:

erk4-scipy (variable time step optimized by scipy)
erk4-homemade for dt = 0.01 s, 0.001 s and 0.0001 s

We can see that the homemade output converges towards the scipy result as dt decreases.
I put the code so you can reproduce it ( I reduce Tmax to avoid very large computations).
In [79]: hub = pgm.Hub(model='DampOscillator', preset='Perfect', verb=False)

In [80]: hub.set_dparam(key='Tmax', value=4, verb=False)

In [81]: hub.run(solver='eRK4-scipy')

In [82]: dax = hub.plot(key='Energy', color='k')

In [83]: hub.run(solver='eRK4-homemade')

In [84]: dax = hub.plot(key='Energy', color='r', dax=dax)

In [85]: hub.set_dparam(key='dt', value=0.001, verb=False)

In [86]: hub.run(solver='eRK4-homemade')

In [87]: dax = hub.plot(key='Energy', color='b', dax=dax)

In [88]: hub.set_dparam(key='dt', value=0.0001, verb=False)

In [89]: hub.run(solver='eRK4-homemade')

In [90]: dax = hub.plot(key='Energy', color='g', dax=dax)

This is an aspect that we had already discussed here, in PR #95
On my opinion, it shows that we should:

Recommend eRK4-scipy by default
Learn how to optimize the time steps size to improve eRK4-homemade

I am still not sure why eRK8-scipy gives wrong results though

Solved, see PR #130

I m closing this issue as it was solved


Attributes on * were data bound prior to Polymer upgrading the element

See https://github.com/dart-lang/polymer-dart/issues/33

@markhats are  you able to post a repro? Most likely this is due to elements being registered out of order. If you open up your web inspector and look at the *.web_components.bootstrap.dart file you can see the order in which elements are being registered (look for CustomElementProxy and CustomTag).

OK -  I think I've got a reasonable repro now. I couldn't seem to reproduce it on a small app so I stripped my app back to somewhere near the minimum required for the issue to show:
Test Repro
If you run the app, you will see the following error generated:
Attributes on slider-control were data bound prior to Polymer upgrading the element. This may result in incorrect binding types. (:1)
This becomes a problem once there are some bindings in place. Uncommenting the 'slider.dart' import in components/popover/panes/line2dpane.dart seems to get rid of the error for some reason but, as far as I can see, there is no reason to need that import. I guess this fix may be similar to the workaround you mentioned previously, whereby all components are imported on the Dart side.

great, I will take a look.

Ok, so this is being caused by a cyclical dependency. The file popoverobjectpane.dart imports ../../autograph.dart, which then imports popoverobjectpane.dart via the import to components/popover/panes/line2dpane.dart. When there are these cyclical dependencies then it is not possible for polymer to know in what order it should register things, and in your case it chooses wrong. There is a warning in the logs about this:
[Warning from polymer (PolymerBootstrapTransformer) on Autograph|web/autograph.html]:
We have detected a cycle in your import graph when running initializers on Line2DPane. This
means the super class PopoverObjectPane has a dependency on this library (possibly
transitive).asset:Autograph/web/components/controls/slider.dart

This warning isn't properly making its way to the warning dialog on the front page though, so I will look into that.
In general, this is caused by the use of parts in autograph.dart. I would just switch it to use imports/exports and then directly include the AxesObject and AxesObjectDrawable classes from their original dart files (in popoverobjectpane.dart).
With polymer in general, its a good idea to avoid the use of parts. Things behave most consistently if every single polymer element has its own library.

OK - thanks.
So should every file/class (even non-polymer elements) ideally be a separate library then, if it can possibly be imported by a polymer element?

It is really just files which contain @CustomTag annotations on classes. By each of these living in its own library it makes it a lot easier to avoid cyclical dependencies.

marking as closed for now, but feel free to re-open if needed

Also its worth noting that using show/hide does not help this situation. Polymer has to read all CustomTag annotations on all classes of any library you import, regardless of if they are visible or not by your actual file. You might only care about class A, but polymer doesn't know whether or not A actually depends on B in the same library, so it has to assume it does.

In my case, the AxesObject and AxesObjectDrawable classes that are causing the problem do not have the @CustomTag annotation as they are not polymer elements. They are really just 'model' classes contained in the 'autograph' library. However, since the 'autograph' library also uses the line2dpane custom element that imports these classes, I am getting the cyclical dependency.
I guess I did sort of think that using show would prevent any cyclic situations. However, as you say, it doesn't. I think the only solution will be to take my model classes out of the 'autograph' library and put them into a separate library or libraries.
I wonder why this wasn't an issue pre-polymer 0.16.0?

Kind of a long story, but basically the way your application is crawled now is completely different. Before we found @CustomTag annotations by looking at script tags found in your html imports, and we would only look in the top level library there (which had its own issues). Now we crawl all your dart imports as well (since you no longer have to have manual html imports).

And yes, you could still use parts for all your files which don't contain @CustomElement annotations.

Not though if they are model class files imported by polymer elements, such AxesObject and AxesObjectDrawable. These don't have @CustomTag annotations but still can't be 'part of' files it seems.

The reason in your particular case is that the library which they are a part of is importing some custom elements (which are then importing the original library). Basically, you could separate that out so there is a library which only has imports/parts which are not custom elements. Then you can export that library from your main library to expose it as part of your main api, and import it in your custom element dart files so they have access to the shared models/etc.

To clarify, heres a really quick example of a possible file layout you could use, https://gist.github.com/jakemac53/85b53ae30e3e766dd3d2

Thanks - that's super helpful!


referential_integrity_check improvements

Add created_at, updated_at to referential_integrity_check
Add aliases/rename aliases to clarify that they're the fields in the table we have data for, not the one that's missing the data
Order referential_integrity_check more helpfully

Approved on my end


Option to use a split instead of a floating window.

I'd love an option to have a live-updating preview in a split window. For example, in the below screenshot, Glow preview would go on the right side.

@mrjones2014 that's already in the future plans. We've already opened a discussion here to check possible implementations. Feel free to add your comments there


Order of the Posts

Problem:
Currently, when a user creates a new post then, it will be shown at the bottom.
Solution:
By default, new posts should be shown at the top.
Additional Feature:
We can also add the sort feature.
Would you like to work on this issue?


App Complete Auth - Teams OAUTH Prompt not showing

Hi
I have really been struggling to get SSO Auth working in my teams bot using the nodejs sample. The provisioning creates an Azure Bot Resource with its linked App Registration and all relevant settings.
The .env and .env.local files get populated with the provisioned resources. I noticed that even though the README states that you need to create an additional app registration and set the MicrosoftAppId, MicrosoftAppPassword etc , the provisioning step has already populated these with the same resources as the  e..g AAD_APP_CLIENT_ID == MicrosoftAppId
I therefore just went with this setup where a single app registration is used. I assume that is correct and can work?
Now the results:
In teams itself (Debugging from VS Studio) , the 'login' request is sent and the MainDialog  responds with buttons , one being 'AAD SSO authentication' , clicking send the 'sso' command....
At this point I would expect the SsoOAuthPrompt to show a sign in button or validate by current creds.
But nothing happens, tracing to MainDIalogue..promptStep --> stepContext.beginDialog(SSOAUTH);
But at this point a callback seems to be made back into the bot,  where the context.TurnContext.activity name is 'signin/tokenExchange'
But then it all dies from there and no further activity or responses.
Interesting to note that the Test in web chat seems to work
Any help will be appreciated, I have tries many posts and combos without luck.
Thanks

I think the issue is that the 'signin/tokenExchange' , activity is comes in with the token value set,  TeamsBot.onTokenResponseEvent is never executed.

@hansliebenberg - Thanks for reporting your issue. Could you please share the sample you are trying out so that we can try it on our end?

Hi,
It is literally the /Microsoft-Teams-Samples/samples/app-complete-auth/nodejs
Thanks

I added soe debug info to index.js:
async function handleBotLogic(context) {
await bot.run(context);
}
// Listen for incoming requests.
server.post('/api/messages', async (req, res) => {
//check if req.body json has a name of "signin/tokenExchange", and clog accoringly
if (req.body.name === "signin/tokenExchange") {
console.log("Token exchange request received");
}
await adapter.process(req, res, handleBotLogic);
});
In the case of a "signin/tokenExchange" , the  adapter.process never invokes the logic function handleBotLogic

@hansliebenberg - Thanks for the update. We are trying this out, we will update you soon.

@hansliebenberg - The sample team is working on the App Complete Auth sample to fix the issue. Meanwhile, could you please try this similar sample?
Microsoft-Teams-Samples/samples/bot-conversation-sso-quickstart/js at main · OfficeDev/Microsoft-Teams-Samples


Addressing 0 length strings.

@sc0ttbeardsley this addresses the issue we see with 0 length strings... somehow std::map cracks with 0 length strings...

thx

CLA is valid!