Have to unplug/plug my card reader after flashing each card.

Etcher version: 1.0.0-beta.15
Operating system and architecture: Win10 x64

I'm using an old USB2 transcend multicard reader. It seems like etcher ejects the whole µSD reader device instead of the card as I have to unplug/replug the card reader to make a new card available.

Hey there,
Etcher ejects drives upon completion by default, but you can disable such feature by un-ticking "Unmount on success" on the settings screen (the cog icon in the top right corner of the application).
Regarding unmounting the SDCard but not the reader, I don't think this can be accomplished, since they both represent the same physical drive, but please re-open if you find a way to do so!

Different USB card readers also work in different ways too... some readers appear as 'empty drives' when no card is inserted, and others only appear to the host OS after a card has been inserted. Some readers with two slots (e.g. a full-size and micro SD) appear as two separate drives (which can be written independently), others only appear as a single drive (i.e. only one slot can be used at once).
So it's quite believable that some card readers will actually need to be physically unplugged, whereas others will only need to have a different card inserted.
@jviotti Should the "Unmount on success" option be renamed to "Eject on success" if that's what it's actually doing? (I dunno about other OSes, but on Linux 'unmount' and 'eject' are different things - a partition that's been unmounted can later be remounted, but a disk that's been ejected is non-functional until it's been removed and reinserted)

@jviotti Should the "Unmount on success" option be renamed to "Eject on success" if that's what it's actually doing? (I dunno about other OSes, but on Linux 'unmount' and 'eject' are different things - a partition that's been unmounted can later be remounted, but a disk that's been ejected is non-functional until it's been removed and reinserted)

In Windows, "unmounting" basically means "ejecting". In UNIX based systems, unmounted drives are still accessible through direct I/O, however in Windows, if you "unmount" a drive, you have to re-plug or "mount" again to be able to access it.

It's worth noting that the term "eject" is likely going to be understood by Windows users, while "unmount" probably won't have any significance to the majority of them.

Makes a lot of sense. Lets change the wording to "Eject" when running on Windows.

Concerning my initial issue, I was reporting this because the behaviour of the ejecting/unmounting from etcher seems different than the ejecting from windows explorer.
If there's anything I can do like spying with a debug tool or else, let me know ;)

Hi @Tryum,
We're using RemoveDrive.exe to handle this (see http://www.uwe-sieber.de/drivetools_e.html). If you could download the tool and experiment with its options that would be amazing. If you find a combination that leads to the results you're expecting, happy to make such change in Etcher!

I'll re-open this so we can investigate together!

Just wanted to mention: I am using a 2013 MacBook with Windows 10 (bootcamp). Using Etcher with the unmout setting enabled will 'remove' the card reader completely forcing you to reboot the system to get it back. Not sure, but maybe the unmount option should be disabled by default until this is fixed? You could also present a dialog at the end "Done, do you want to eject? [yes] / [no]".

ooh, that sounds annoying indeed. maybe we should hold off from ejecting
until that is either fixed, or at least we can divide the cases where we're
sure we're doing the right thing and those where we're not.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1<PHONE_NUMBER>
@alexandrosm
On Thu, Dec 8, 2016 at 7:19 AM, Beatnukem<EMAIL_ADDRESS>wrote:

Just wanted to mention: I am using a 2013 MacBook with Windows 10
(bootcamp). Using Etcher with the unmout setting enabled will 'remove' the
card reader completely forcing you to reboot the system to get it back. Not
sure, but maybe the unmount option should be disabled by default until this
is fixed? You could also present a dialog at the end "Done, do you want to
eject? [yes] / [no]".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/750#issuecomment-265670636,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCMY9Z18c114ssN9uHgI5YW_jvfmBks5rF691gaJpZM4KPTg8
.

@Beatnukem I assume this is the internal card-reader slot on the laptop you're talking about?
Can you try doing the same thing in Etcher under OSX, to see if it exhibits the same behaviour? (or whether it's just a Windows-specific thing)

@Beatnukem

Using Etcher with the unmout setting enabled will 'remove' the card reader completely forcing you to reboot the system to get it back. Not sure, but maybe the unmount option should be disabled by default until this is fixed?

In the Windows world, "unmounting" means "ejecting", and there is no way to unmount a drive like in the UNIX world while still be able to open a file descriptor to it (unless you remove the drive letters from it, but that's a mess, since you'll have to manually assign it back in order to use the drive, since not even a reboot will work).
Since this is an internal SDCard reader, there is indeed no way to plug it back :/
This is the tool we use to eject drives on Windows: http://www.uwe-sieber.de/drivetools_e.html. Maybe you can play around with the options it provides to see if you can get it to do the right thing?
RemoveDrive DriveSpec: [-v][-d][-l][-t][-h][-a][-b][-i][-s][-w:nnnn]

DriveSpec  is the drive to remove, e.g. U: or an NTFS mountpoint as
           "C:\CardReader\Multi Reader SD"
           or . for the current drive
           or \ for the drive of the RemoveDrive.exe
           or a volume name like \\?\Volume{433619ed-c6ea-11d9-a3b2-806d6172696f}
           or a kernel name like \Device\HarddiskVolume2 (Volume)
           or a kernel name like \Device\Harddisk2\DR0 (Disk)
           or a partition name like \Device\Harddisk2\Partition1
           or a device ID
           or a friendly name like 'Corsair Voyager'
           or a volume label like 'Backup01'
           Wildcards can be used.
[-47]      perform safe removal resulting in problem code 47
[-v]       remove the volume only instead of the drive or whole device
[-d]       remove the drive only instead of the whole device
[-L]       loop until success
[-e]       try to dismount and eject if the removal fails
[-t]       release TrueCrypt volumes hosted by the drive to remove
[-h]       show open handles (admin rights required, exprimental, may freeze!)
[-h:X]     show open handles on a different drive (e.g. a TrueCrypt
           volume whose container is on the drive to remove, not required
           if -t -h is used)
[-a]       activates Windows of applications owning the open handles, requires -h
[-w:nnnn]  wait nnnn milliseconds before closing the program (time to read its output)
[-s]       self delete removedrive.exe
[-b]       let Windows show the "Safe To Remove Hardware" balloon tip
           (may lead to long delays)
[-i]       stop Windows indexing service (CiSvc) for a moment if required (admins)
[-dbg]     show debug information

I assume other internal SD card readers on Windows laptops don't have this same problem too? I wonder what the difference is here? Hardware-level problem, or Windows driver-level problem? (i.e. I wonder if a different driver might fix this?)

Well I guess it's exactly the same problem that I have, hower my reader
being external and USB, I can disconnect/reconnect it to get it back
without rebooting.
I think there is a difference between ejecting the volume and ejecting the
device, and etcher seems to eject the whole device.
Thibault Jochem
Software Engineer
EFREI Promo 2007
On Thu, Dec 8, 2016 at 4:56 PM, Andrew Scheller<EMAIL_ADDRESS>wrote:

I assume other internal SD card readers on Windows laptops don't have
this same problem too? I wonder what the difference is here? Hardware-level
problem, or Windows driver-level problem? (i.e. I wonder if a different
driver might fix this?)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/750#issuecomment-265775594,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJ3WTvHcvN4WsiY70IsM57E2wEn5OHcks5rGCi2gaJpZM4KPTg8
.

I assume there'd be a lot more problem reports, if everyone who has a Windows laptop with an internal SD-reader who uses Etcher, had to reboot their PC before they were able to use their SD slot again!!

Hello @jviotti, @lurch
I have a similar issue with a internal multi-card reader that was reported in thread you mentioned here above (Error: Path must be a string. Received undefined). Basically, once the SD card is burned and ejected, there is no way to re-use the drives. A computer reboot is necessary.
In the meantime, I learned there is an option to not eject the card.
I am running Windows 7 French and here is the info I found about my reader:
From http://driverlibs.com/download/General-UDisk-USB-Device
HardwareIDs:
USBSTOR\DiskGeneric-Compact_Flash___1.01
USBSTOR\DiskGeneric-Compact_Flash___
USBSTOR\DiskGeneric-
USBSTOR\Generic-Compact_Flash___1
Generic-Compact_Flash___1
USBSTOR\GenDisk
GenDisk
Driver: 6.1.7601.19133
Hope that helps, if not let me know if you need more info.
Thanks

Hi @daturach ,
What happens when you plug an SDCard to your PC using such adapter? If you open My PC, do you see various "unassigned" drive letter apart from the one from your SD Card?
When you open Etcher, and open the drive selector modal, how many drive letters does it list for that drive? Are they correct, or is Etcher listing those "unassigned" letters?
Check my previous comment (https://github.com/resin-io/etcher/issues/750#issuecomment-265764658) about the CLI tool we're using to handle unmounting.
They have some particularly interesting options that it might be worth trying:
[-v]       remove the volume only instead of the drive or whole device
[-d]       remove the drive only instead of the whole device

If your SDCard is assigned to E:, you could try:
> RemoveDrive.exe E: -v

And:
> RemoveDrive.exe E: -d

To see if any makes a difference.

Hello @jviotti
SD card is connected to "h:"

A propos, could you make this dialog box resizable? :-)
Thanks for the tool. Will have a look later this week.

Hi @daturach ,
We have some designs to make this larger, and possible stop making it a modal in the future, which should resolve this problem. We'll share the designs very soon, and you can provide feedback if you like.
Thanks a lot, looking forward to hear how it went! In any case, I'm trying to purchase an adapter similar to the one you have, and see if I manage to reproduce here as well.

Hello @jviotti
Both tests required a PC reboot to get the H: drive back but they return different messages.
Only H: is not available. I:, J: and K: are still there.
removedrive h: -d
RemoveDrive V2.8.4 (x64) - prepares drives for safe removal
Freeware by Uwe Sieber - www.uwe-sieber.de
Removing Drive 'Generic- SD/MMC USB Device' (H:)
success  (reactivation is possible)
removedrive h: -v
RemoveDrive V2.8.4 (x64) - prepares drives for safe removal
Freeware by Uwe Sieber - www.uwe-sieber.de
Removing Volume H:
success

@daturach  For both tests, after running removedrive I assume you tried physically removing and then re-inserting the SD card a couple of times?
Does the "safely remove" option in the Windows systray also exhibit this "drive becomes unusable until you reboot" behaviour, or does it do something different?
@jviotti This search suggests that Alcor Card Readers need a specific software driver installed, which suggests they behave differently from "standard" USB card readers (which don't require any additional drivers to be installed), so maybe that's why RemoveDrive.exe is giving us this "weird" behaviour for these really old USB card readers? (and if my assumption is correct and it only affects old card readers, I guess that would explain why this problem is so 'rare')

@daturach Thanks a lot for the testing! So to make it clearer, both with -d and -v only the correct drive was ejected but not the others from the same reader, right (which is the expected behaviour)?

Both tests required a PC reboot to get the H: drive back

That's odd. I wonder if passing -47 along with these options result in the correct behaviour:
[-47]      perform safe removal resulting in problem code 47

So -47 -v would perform a "safe removal" of the "volume only".
@lurch Could be. I'd be interred to know how the "safely remove" option in the systray compares to the behaviour we see on RemoveDrive.exe.

@lurch: Yep, I did unplug the card.
@jviotti: yes, only the correct drive.
Will verify later if [-47] does the correct job.

OK, its at least an improvement. I'll wait to heard how -47 went before sending a PR.

Ok, here we go:
removedrive h: -47 -v
RemoveDrive V2.8.4 (x64) - prepares drives for safe removal
Freeware by Uwe Sieber - www.uwe-sieber.de
Removing Volume H:
failed
removedrive h: -47 -d
RemoveDrive V2.8.4 (x64) - prepares drives for safe removal
Freeware by Uwe Sieber - www.uwe-sieber.de
Removing Drive 'Generic- SD/MMC USB Device' (H:)
Veto=IllegalDeviceRequest
failed
Ok, tomorrow, I am going to buy a new computer...Just joking :-)

Haha, that sucks. I wonder why that fails. I remember I got it running on Windows 10 at some points. I just got Windows 7, so I'll see if this is a Windows 7 only issue.

@daturach Just to re-request - Does the "safely remove" option in the Windows systray also exhibit this "drive becomes unusable until you reboot" behaviour, or does it do something different?
(i.e. how do you normally "eject" a card from your SD reader, without causing the whole drive to disappear?)

@lurch To answer your question, I do a right-click on the icon on the systray and then eject.

I have the same issue on a fairly new Lenovo desktop with built in card reader. Lenovo K450e (Model 10181). I was not able to find a set of RemoveDrive options that work. Eject from the windows menu works just fine.

@jviotti doesn't diskpart have functionality to eject drives? Is there a reason you can't use that instead of RemoveDrive.exe?

@jviotti doesn't diskpart have functionality to eject drives? Is there a reason you can't use that instead of RemoveDrive.exe?

I don't know, but looks like something worth investigating. I'll do so and report back soon.

I found this: http://blog.mundy.co/kb/28-windows/505-safely-remove-a-disk-using-the-command-line

Type "select volume ", where  is the number of your USB storage.
Then, type "remove all dismount". Type "exit" to leave diskpart. Enjoy! ;)

However this removes the associated drive letter of the drive, meaning that you won't get Windows to recognize it back when you plug it again unless you manually assign a new drive letter to it (so its not really the same as the "Safely Remove" Windows feature).
I Googled for alternatives and found the following pages:

http://superuser.com/questions/443162/remove-usb-device-from-command-line
http://stackoverflow.com/questions/22705830/safely-remove-usb-from-batch-file

They all seem to recommend the RemoveDrive.exe file we're already using :(

The same page which hosts the RemoveDrive utility, also hosts an EjectMedia utility which it describes as "Ejecting the media only is especially suitable for internal cardreaders".
Is that something you've experimented with yet @jviotti ? (although if that does work, I obviously have no idea how you'd know when to use RemoveDrive and when to use EjectMedia!)
Looks like this might have some useful info too...

Check this: https://www.codeproject.com/articles/259577/how-to-flush-a-storage-volumes-the-file-cache-lock. A good explanation of how EjectMedia works. It looks like we have enough technical information to go and implement our own unmounting program.

OK, I think I found something that works. @daturach @Tryum Can you try the EjectMedia tool calling it like this?
EjectMedia.exe X: -f

Obviously replacing X: with your corresponding drive letter.
I've tested this on the internal reader of a MacBook Pro running BootCamp, on an USB Hub, on

Sorry for the open/close. I've hit the wrong button :)

Not passing the -f option causes the internal reader to remain active on My PC even though its empty.

Ok @jviotti . I got the same results as you and I don't need to reboot anymore! Well done
EjectMedia.exe H: -f

EjectMedia V2.7.2 (x64) - ejects a media from a drive
Freeware by Uwe Sieber - www.uwe-sieber.de
Ejecting media from H: (Generic- SD/MMC USB Device)
Flushing cache... OK
Allowing eject... OK
Locking volume... OK
Dismounting volume... OK
Ejecting media... OK
success

Will EjectMedia work with any drives ? Internal or external?
Without the -f option, same as above:
EjectMedia.exe H:

EjectMedia V2.7.2 (x64) - ejects a media from a drive
Freeware by Uwe Sieber - www.uwe-sieber.de
Ejecting media from H: (Generic- SD/MMC USB Device)
Flushing cache... OK
Allowing eject... OK
Locking volume... OK
Dismounting volume... OK
Ejecting media... OK
success

Amazing news @daturach ! As far as my experimentations went, this new tool works fine for both internal and external drives. Weird that omitting -f yields the same result to you. I'll keep it just in case anyway.
I'll be sending a PR to make use of this tool very soon.

Just for reference, I double checked that we're allowed to re-distribute this:
From http://www.uwe-sieber.de/drivetools_e.html

Allowed:

usage in any environment, including commercial
include in software products, including commercial
include on CD/DVD of computer magazines

Since we know how this tool works, we might re-implement it and extend it ourselves sometime in the future anyway.
@jhermsmeier ^

So just to clarify, are you saying that in hindsight we should have just used EjectMedia.exe instead of RemoveDrive.exe all along? ;)
If you haven't already done so, it might be worth checking to see what EjectMedia does when you try to use it on a USB HDD as opposed to a USB flash drive?

Looks like you're right. EjectMedia works on USB drives but not on USB HDD (it does nothing in that case). I think we can try EjectMedia, and if the child process exits with code 0 but still drivelist detects that the drive is still mounted, then we fallback to RemoveDrive. What do you think?

Sounds reasonable, if sufficient testing shows that it works as expected? Would we need to insert some kind of delay between running EjectMedia and re-scanning the driveslist in case it takes a couple of seconds for the drive to unmount / disappear? (or is that guaranteed to have happened by the time EjectMedia exits?)
And that raises an interesting side-question - as well as returning the info for all drives, would it be worth adding a mode to drivelist to return information only for a specific drive? (in case that's quicker than scanning all drives)

And that raises an interesting side-question - as well as returning the info for all drives, would it be worth adding a mode to drivelist to return information only for a specific drive? (in case that's quicker than scanning all drives)

This is an amazing idea. Updating the scripts to handle both a single drive or all drives might be tricky, but we can start by exposing a function to get information about a single one that simply runs the script and filters for the one you're interested in, and optimize later.


Workaround for an error found on stage.

Workaround for an error found on stage.
Closes OSIDB-1369

This is a follow-up to https://github.com/RedHatProductSecurity/osidb/pull/344

Superseded by #347


Output PNG files gets the wrong size / content

Repro step:

I've done pip install numpy scipy matplotlib pypng
I called GenerateBlueNoiseDatabase(range(1), 32, 32, range(1,1), 1.9) in BlueNoise.py
I'm using Windows 10. Tried python 2.7 and 3.8.0.

Problem:

The LDR-texture that comes out is not square, it has a fourth of the intended width and has the wrong content too

All other attempts at saving images using StoreNoiseTextureLDR  gets me the same issue. It seems like the np.dstack is ignored or something like that.
My questions:

Is the pypng the right import for the png module?
Has your script ever been tested on Windows?

If I add the code below to the end of StoreNoiseTextureLDR it seems to work on my computer:
w = Texture.shape[1]
h = Texture.shape[0]
SaveTexture = np.zeros((h, w * 4), dtype = np.uint8)

for y in range(h):
	for x in range(w):
		for c in range(4):
			SaveTexture[y, x * 4 + c] = Texture[y, x, c]

# Save the image
png.from_array(SaveTexture,"RGBA;8").save(OutputPNGFilePath);

Thanks for the nice noise textures!

Apparently the pypng interface changed a bit at some point in the past three years. It should be fixed now. Thanks for the report.


tutorial island ui + pathing fix

pathing no longer gets stuck at rat gates or western anvil etc. also added small pauses as antiban/more human-like inputs.
uses sprites to find interface widgets in order to have compatibility across interface styles. very stopgap measure! but clickFlashyWidget() could be useful in Rs2Widget anyway (maybe with a different name)

The build failed, please take a look at it


String has no match

Perhaps I'm confused but without this method, does derw have the capacity to match regular expressions?

Hi!
So, basically you can use methods on objects in Derw as you would in TypeScript. This means you can use something like:
aString: string
aString =
    ""

someTest: boolean
someTest =
    let
        match: any
        match =
            aString.match "[a-z]*"
    in
        match == null

or do whatever you want with the match array.
If you were going to using it more often though, the way would be to create a kernel code wrapper around the regex functions to make it more like Derw (for example - not using nulls and return a Maybe instead). For more examples of kernel code, check out this repo - and the GitBook, where everything has examples of what code gets compiled to. Pull requests welcome!
Feel free to reach out to me on Twitter or Zulip if you have any other questions.


Nested foreach do not seem to be possible

Hello,
Looking at the documentation, it is said that we can create nested foreach inside a flow.
I have been trying several combinations to not get an error. My last try is like this :

class ForeachFlow(FlowSpec):

    @step
    def start(self):
        self.titles = ['Stranger Things',
                       'House of Cards',
                       'Narcos']
        self.next(self.a, foreach='titles')

    @step
    def a(self):
        self.title = '%s processed' % self.input
        self.next(self.b, foreach='title')

    @step
    def b(self):
        self.char = self.input
        self.next(self.join)

    @step
    def join(self, inputs):
        merged = self.merge_artifacts(inputs)
        print(merged)

        self.next(self.end)

    @step
    def end(self):
        print('end!')

if __name__ == '__main__':
    ForeachFlow()

I keep getting an error which looks like this :
Validating your flow...
    Validity checker found an issue:
    Step end reached before a split started at step(s) a were joined. Add a join step before end.

I don't know how to properly merge everything right after the second-level of nested foreach branch, without getting an error.
The documentation does not help about these nested foreach, how are we supposed to do it?

Hello.
Each split must be joined so something like this would work:
class ForeachFlow(FlowSpec):

    @step
    def start(self):
        self.titles = ['Stranger Things',
                       'House of Cards',
                       'Narcos']
        self.next(self.a, foreach='titles')

    @step
    def a(self):
        self.title = '%s processed' % self.input
        self.next(self.b, foreach='title')

    @step
    def b(self):
        self.char = self.input
        self.next(self.join_b)

    @step
    def join_b(self, inputs):
        # You can't use merge_artifacts here because your artifacts are not mergeable
        # since Metaflow can't "guess" which version of it to pick.
        self.list_of_chars = [i.char for i in inputs]
        self.merge_artifacts(inputs, include=["title"]) # This just merges title -- showing how to use it
        print("Merging all bs, found characters: %s" % str(self.list_of_chars))

        self.next(self.join_a)

    @step
    def join_a(self, inputs):
        # You can't use merge_artifacts here because your artifacts are not mergeable
        # since Metaflow can't "guess" which version of it to pick.
        self.chars_per_title = {i.title: i.list_of_chars for i in inputs}
        print("Merged everything: %s" % str(self.chars_per_title))

        self.next(self.end)

    @step
    def end(self):
        print('end!')

if __name__ == '__main__':
    ForeachFlow()

It seems to work like a charm, thanks!
I guess the join_b comment about "You can't use merge_artifacts here..." is because of a copy paste, right?

Sorry, I should have made the comment better. I meant to say that you can't use the standard merge_artifacts like you had because there are "conflicting" artifacts. Here I use it with include and restrict it to the artifacts I know are the same so that is ok.


Hide when no maxlength set

is it possible to hide the badge when there is no maxlength set on the input item? it is showing NaN/undefined. There are some inputs where I do not want to set a max length.

[maxlength]="toNumber(myNum)" NaN will be 0


error when running train.ipynb

Hi, I got below error messages when I run train.ipynb, can you help to take a look>
model.fit(train_ds,
      epochs=20,

      steps_per_epoch=train_steps_per_epoch,

      validation_data=val_ds,

      validation_steps=val_steps_per_epoch,

      initial_epoch=0,

      callbacks = callbacks)

TypeError                                 Traceback (most recent call last)
 in
5           validation_steps=val_steps_per_epoch,
6           initial_epoch=0,
----> 7           callbacks = callbacks)
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
823         max_queue_size=max_queue_size,
824         workers=workers,
--> 825         use_multiprocessing=use_multiprocessing)
826
827   def evaluate(self,
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
233           max_queue_size=max_queue_size,
234           workers=workers,
--> 235           use_multiprocessing=use_multiprocessing)
236
237       total_samples = _get_total_number_of_samples(training_data_adapter)
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_training_inputs(model, x, y, batch_size, epochs, sample_weights, class_weights, steps_per_epoch, validation_split, validation_data, validation_steps, shuffle, distribution_strategy, max_queue_size, workers, use_multiprocessing)
591         max_queue_size=max_queue_size,
592         workers=workers,
--> 593         use_multiprocessing=use_multiprocessing)
594     val_adapter = None
595     if validation_data:
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_inputs(model, mode, x, y, batch_size, epochs, sample_weights, class_weights, shuffle, steps, distribution_strategy, max_queue_size, workers, use_multiprocessing)
704       max_queue_size=max_queue_size,
705       workers=workers,
--> 706       use_multiprocessing=use_multiprocessing)
707
708   return adapter
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/data_adapter.py in init(self, x, y, sample_weights, standardize_function, **kwargs)
700
701     if standardize_function is not None:
--> 702       x = standardize_function(x)
703
704     # Note that the dataset instance is immutable, its fine to reusing the user
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in standardize_function(dataset)
658         model.sample_weight_mode = getattr(model, 'sample_weight_mode', None)
659
--> 660       standardize(dataset, extract_tensors_from_dataset=False)
661
662       # Then we map using only the tensor standardization portion.
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split, shuffle, extract_tensors_from_dataset)
2376     is_compile_called = False
2377     if not self._is_compiled and self.optimizer:
-> 2378       self._compile_from_inputs(all_inputs, y_input, x, y)
2379       is_compile_called = True
2380
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _compile_from_inputs(self, all_inputs, target, orig_inputs, orig_target)
2634         sample_weight_mode=self.sample_weight_mode,
2635         run_eagerly=self.run_eagerly,
-> 2636         experimental_run_tf_function=self._experimental_run_tf_function)
2637
2638   # TODO(omalleyt): Consider changing to a more descriptive function name.
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
455     self._self_setattr_tracking = False  # pylint: disable=protected-access
456     try:
--> 457       result = method(self, *args, **kwargs)
458     finally:
459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, distribute, **kwargs)
437           targets=self._targets,
438           skip_target_masks=self._prepare_skip_target_masks(),
--> 439           masks=self._prepare_output_masks())
440
441       # Prepare sample weight modes. List with the same length as model outputs.
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _handle_metrics(self, outputs, targets, skip_target_masks, sample_weights, masks, return_weighted_metrics, return_weighted_and_unweighted_metrics)
2020           metric_results.extend(
2021               self._handle_per_output_metrics(self._per_output_metrics[i],
-> 2022                                               target, output, output_mask))
2023         if return_weighted_and_unweighted_metrics or return_weighted_metrics:
2024           metric_results.extend(
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _handle_per_output_metrics(self, metrics_dict, y_true, y_pred, mask, weights)
1971       with K.name_scope(metric_name):
1972         metric_result = training_utils.call_metric_function(
-> 1973             metric_fn, y_true, y_pred, weights=weights, mask=mask)
1974         metric_results.append(metric_result)
1975     return metric_results
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_utils.py in call_metric_function(metric_fn, y_true, y_pred, weights, mask)
1153
1154   if y_pred is not None:
-> 1155     return metric_fn(y_true, y_pred, sample_weight=weights)
1156   # Mean metric only takes a single value.
1157   return metric_fn(y_true, sample_weight=weights)
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/metrics.py in call(self, *args, **kwargs)
194     from tensorflow.python.keras.distribute import distributed_training_utils  # pylint:disable=g-import-not-at-top
195     return distributed_training_utils.call_replica_local_fn(
--> 196         replica_local_fn, *args, **kwargs)
197
198   @property
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/distribute/distributed_training_utils.py in call_replica_local_fn(fn, *args, **kwargs)
1133     with strategy.scope():
1134       return strategy.extended.call_for_each_replica(fn, args, kwargs)
-> 1135   return fn(*args, **kwargs)
1136
1137
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/metrics.py in replica_local_fn(*args, **kwargs)
177     def replica_local_fn(*args, **kwargs):
178       """Updates the state of the metric in a replica-local context."""
--> 179       update_op = self.update_state(*args, **kwargs)  # pylint: disable=not-callable
180       with ops.control_dependencies([update_op]):
181         result_t = self.result()  # pylint: disable=not-callable
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/utils/metrics_utils.py in decorated(metric_obj, *args, **kwargs)
74
75     with tf_utils.graph_context_for_symbolic_tensors(*args, **kwargs):
---> 76       update_op = update_state_fn(*args, **kwargs)
77     if update_op is not None:  # update_op will be None in eager execution.
78       metric_obj.add_update(update_op)
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py in call(self, *args, **kwds)
566         xla_context.Exit()
567     else:
--> 568       result = self._call(*args, **kwds)
569
570     if tracing_count == self._get_tracing_count():
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py in _call(self, *args, **kwds)
613       # This is the first call of call, so we have to initialize.
614       initializers = []
--> 615       self._initialize(args, kwds, add_initializers_to=initializers)
616     finally:
617       # At this point we know that the initialization is complete (or less
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
495     self._concrete_stateful_fn = (
496         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
--> 497             *args, **kwds))
498
499     def invalid_creator_scope(*unused_args, **unused_kwds):
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
2387       args, kwargs = None, None
2388     with self._lock:
-> 2389       graph_function, _, _ = self._maybe_define_function(args, kwargs)
2390     return graph_function
2391
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py in _maybe_define_function(self, args, kwargs)
2701
2702       self._function_cache.missed.add(call_context_key)
-> 2703       graph_function = self._create_graph_function(args, kwargs)
2704       self._function_cache.primary[cache_key] = graph_function
2705       return graph_function, args, kwargs
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
2591             arg_names=arg_names,
2592             override_flat_arg_shapes=override_flat_arg_shapes,
-> 2593             capture_by_value=self._capture_by_value),
2594         self._function_attributes,
2595         # Tell the ConcreteFunction to clean up its graph once it goes out of
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
976                                           converted_func)
977
--> 978       func_outputs = python_func(*func_args, **func_kwargs)
979
980       # invariant: func_outputs contains only Tensors, CompositeTensors,
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py in wrapped_fn(*args, **kwds)
437         # wrapped allows AutoGraph to swap in a converted function. We give
438         # the function a weak reference to itself to avoid a reference cycle.
--> 439         return weak_wrapped_fn().wrapped(*args, **kwds)
440     weak_wrapped_fn = weakref.ref(wrapped_fn)
441
~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/framework/func_graph.py in wrapper(*args, **kwargs)
966           except Exception as e:  # pylint:disable=broad-except
967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
969             else:
970               raise
TypeError: in converted code:
/home/ec2-user/SageMaker/crnn_by_tensorflow2.2.0/models/accuracy.py:24 update_state  *
    y_true = tf.sparse.reset_shape(y_true, [b, max_width])
/home/ec2-user/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/ops/sparse_ops.py:1827 sparse_reset_shape
    sp_input = _convert_to_sparse_tensor(sp_input)
/home/ec2-user/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/ops/sparse_ops.py:69 _convert_to_sparse_tensor
    raise TypeError("Input must be a SparseTensor.")

TypeError: Input must be a SparseTensor.

Looks like above is due to TensorFlow version, I changed the version based on requirements.txt, and got same error message from your sample repo, can you help on this?
`Epoch 1/20
2020-09-16 02:59:12.522902: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-09-16 02:59:20.987688: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-09-16 02:59:44.767544: E tensorflow/stream_executor/cuda/cuda_dnn.cc:319] Loaded runtime CuDNN library: 7.5.1 but source was compiled with: 7.6.4.  CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
2020-09-16 02:59:44.769539: E tensorflow/stream_executor/cuda/cuda_dnn.cc:319] Loaded runtime CuDNN library: 7.5.1 but source was compiled with: 7.6.4.  CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
Traceback (most recent call last):
File "train.py", line 56, in
main()
File "train.py", line 53, in main
callbacks = callbacks)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
return method(self, *args, **kwargs)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 848, in fit
tmp_logs = train_function(iterator)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 580, in call
result = self._call(*args, **kwds)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 644, in _call
return self._stateless_fn(*args, **kwds)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 2420, in call
return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1665, in _filtered_call
self.captured_inputs)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1746, in _call_flat
ctx, args, cancellation_manager=cancellation_manager))
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 598, in call
ctx=ctx)
File "/home/ec2-user/anaconda3/envs/jackie/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
(0) Unknown:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node sequential/conv2d/Conv2D (defined at train.py:53) ]]
[[transpose/_42]]
(1) Unknown:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node sequential/conv2d/Conv2D (defined at train.py:53) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_15461]
Function call stack:
train_function -> train_function`

你好。关于第一个问题的话我也曾经遇到过这个问题，如果你用的是tf2.0.0的话，可以选择把./models/crnn.py中的tf.keras.layers.Conv2D(filters=64, kernel_size=3, padding='same',activation='relu',input_shape=OUTPUT_SHAPE)改为tf.keras.layers.Conv2D(filters=64, kernel_size=3, padding='same',activation='relu')，即去掉input_shape；但是在这种情况下，你第一次运行你依然会报错，你需要在不关闭这个进程的情况下，再执行一次model.fit这一步，我觉得可能是版本原因导致的。关于第二个问题的话，我觉得可能是GPU导致的，你可以先验证下tf-gpu是否能正常使用。


Improve accuracy for searching by name.

(Discussion needed first)
When searching by name, the 5th result of the search will be omitted due to this line:
await spotifyApi.searchTracks(name, {market:'US', limit:1, offset:5})
In my opinion, we should remove the offset and show the 1st result instead. This will result in the most popular song being omitted which statistically speaking should be what the user was searching for.
Examples:
Query: the%20nights (the nights)
Offset 5:

No offset:

Very simple issue to fix, literally just removing a few characters from the line mentioned above in /index.js

I don't know why I kept offset 5 at the start 😕. Maybe I thought of what if the several song with same name are there.
Anyways, I think setting offset to 1 will be fine. As if the user doesn't find his song then he can always use the id param.
You can go ahead. I have assigned you.


file maintainer please do make necessary

As, according to this structure of file, we are not able to add our team details as prescribed by sir. Thanks in advance!

As, according to this structure of file, we are not able to add our team details as prescribed by sir. Thanks in advance!

Much appreciated @utkarshtambe10 thank you for making PullRequest 🚀

@all-contributors please add @utkarshtambe10 for example, maintenance

Yeah sure. Welcome!


Is there a simple and fast way to disable wandb?

S

type wandb offline in your terminal.

thanks , resolved my issue


Add regression tests

It would be nice to not have to worry about breaking everything when making changes.
It's probably not necessary to set up a local IRC channel for this -- since the irc module exports an EventEmitter, we can probably emit events manually to spoof incoming IRC messages.

2 things that'll make this easier:

My node irc server that I'm writing
The sqlite support that I've written and will put up a PR for once #85 is merged

Then we can get a nice bunch of tests and this would make things like #85 much easier to test in the future.


Response times for transactions endpoint are really slow

Core services log response times of around 2seconds, while the client gateway needs around 20 seconds.
This seems to be a result of one of the lates releases and we should indentify why and if possible fix it.

This seems to be related to the extensive fetching of token information from our service.
Quick fix (to be done asap): Increase INFO_CACHE_DURATION (e.g. to 1 day)
Proper fix(es) (to be discussed):

Batch fetching of token information
Warm token information cache


Adding a map related project

A site that shows recovery rate verses active cases of on a map.
Community Contribution
Link to live project, video, or image:
DEMO

Sharing

[x] I would be happy to have my project shared on The Coding Train's social media!

Instagram:
https://www.instagram.com/ashik.paul/
Twitter:
https://twitter.com/ashikpaul19
This project may be silly, but I learnt a lot. Thank you so much @shiffman. Never knew using a map was this easy.

Looks awesome thanks for sharing! ✨🌍✨


[Maps] Add maps-search package

Packages impacted by this PR

maps-search

Issues associated with this PR

NA

Describe the problem that is addressed by this PR

Introduce the maps-search SDK for the Azure Map Search Service

What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?
Are there test cases added in this PR? (If not, why?)

Yes

Provide a list of related PRs (if any)

#22566 : The previous PR contains search, route, and render. We decided to separate to three for easier review.

Command used to generate this PR:**(Applicable only to SDK release request PRs)
Checklists

[x] Added impacted package name to the issue description
[x] Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
[x] Added a changelog (if necessary)

Here is the API view created previously: https://apiview.dev/Assemblies/Review/737c63ff29ea4eb8ad335ddabb4c382f

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-maps-search

/azp run js - maps-search - ci

/azp run js - maps - ci

/azp run js - maps-search - ci

/azp run js - maps - ci

/azp run js - maps - ci

/azp run js - maps-search - ci

/azp run js - maps - ci

/azp run js - maps - ci

/azp run js - maps - ci

/azp run js - maps - ci

/azp run js - maps - ci

/azp run js - maps - ci

/azp run js - maps - tests

/azp run js - maps-search - ci

/azp run js - maps-search - tests

Great job @andykao1213 🔥

/azp run js - maps-search - tests

/azp run js - maps-search - tests

/azp run js - maps-search - tests


Refactor consent components to use CSS variables that match the brandable colour palette from designs

Description
Our latest beautiful consent components designs Figma link use 12 brandable colours to set the styles:

I'd recommend a nice global colour palette of CSS variables to match these like:
--fides-consent-overlay-primary-color (used for "Modal & Banner Outline")
--fides-consent-overlay-primary-button-color (used for "Primary button resting")
--fides-consent-overlay-primary-hover-color (used for "Primary button hover")
--fides-consent-overlay-primary-active-color (used for "Switch active")
--fides-consent-overlay-primary-active-disabled-color (used for "Switch active disabled")
--fides-consent-overlay-inactive-color (used for "Switch inactive")
--fides-consent-overlay-disabled-color (used for "Switch disabled")
--fides-consent-overlay-background-color (used for "Background")
--fides-consent-overlay-row-hover-color (used for "Row hover")
--fides-consent-overlay-row-divider-color (used for "Row divider")
--fides-consent-overlay-title-font-color (used for "Title font")
--fides-consent-overlay-body-font-color (used for "Body font")

Assumptions
Main assumption here is that these should form the core 12 brandable colours for all the components. It's also safe to assume we'll want to make it easy to configure these with a styling editor in the future!
Acceptance Criteria

Banner styles can be customized using the 12-colour palette above
Modal styles can be customized using the 12-colour palette above

Mockups
See Figma links:

Stylesheet: https://www.figma.com/file/dmEwdK3xZwjKfGVQ9t66xe/Fides-v.2-Master-Working-Files?type=design&node-id=6464-134350&t=8104bV4t4W5TBKPv-0
Two alternative mockups using different branding: https://www.figma.com/file/dmEwdK3xZwjKfGVQ9t66xe/Fides-v.2-Master-Working-Files?type=design&node-id=6464-134350&t=ls6XpE1B1Ume7uLk-0

Implementation Notes/Details

moving to done


How to get the detail response of FeedReelsMediaRequest?

public void testName() throws Exception {
    IGClient client = SerializeTestUtil.getClientFromSerialize("igclient.ser", "cookie.ser");
    IGResponse response =
            new FeedReelsMediaRequest("archiveDay:18116528737123990").execute(client).join();
    Assert.assertEquals("ok", response.getStatus());
    log.debug("Success");
}

What do you mean detail response?

the likers info and the commenters info

I don't think reel media has those info.


Automated cherry pick of #943: 避免转换成shell格式后导致合并失败

Cherry pick of #943 on release/2.9.0.
#943: 避免转换成shell格式后导致合并失败

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.9.0@1ef132b). Click here to learn what that means.
The diff coverage is 0%.

@@               Coverage Diff               @@
##             release/2.9.0    #953   +/-   ##
===============================================
  Coverage                 ?   7.54%
===============================================
  Files                    ?     343
  Lines                    ?   53873
  Branches                 ?       0
===============================================
  Hits                     ?    4064
  Misses                   ?   49428
  Partials                 ?     381

Flag
Coverage Δ

#aFlag
7.54% <0%> (?)

Impacted Files
Coverage Δ

pkg/compute/models/guestdrivers.go
18.18% <ø> (ø)

pkg/compute/models/guests.go
0.66% <ø> (ø)

pkg/compute/guestdrivers/openstack.go
9.8% <0%> (ø)

pkg/compute/guestdrivers/aws.go
6.75% <0%> (ø)

pkg/compute/guestdrivers/base.go
0% <0%> (ø)

pkg/compute/guestdrivers/managedvirtual.go
0% <0%> (ø)

pkg/compute/guestdrivers/qcloud.go
5.43% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ef132b...f5fb8e7. Read the comment docs.

/lgtm
/approve


Drop Python 3.9

Remove Python 3.9 from CI tests, and update minimum python in run requirements in conda meta.yaml.
Closes #497

coverage: 55.953% (+0.2%) from 55.76%
when pulling a06896ae2fc48a97862d64394c957ce304932b18 on ACCESS-NRI:497-remove-python-3-9-from-CI-tests
into 78f67ff2ff506fa471a8801d8051b4b2f6132b84 on payu-org:master.

Had to remove 3.9 (and add 3.12) in the branch protections for master.


Add Settings to Terminal Profiles Extension (Core)

Summary of the Pull Request
This PR adds the equivalent settings functionality seen in PT Run Profiles Plugin out of the box.
It leverages the Settings class helpers to create the settings form.
Here are some screenshots of the resulting changes:

A thought: you're using state.json, which is also what the bookmarks builtin uses. That means we end up blowing away the bookmarks.
For now, maybe just rename this to wt_state.json or something. We'll want to be smarter in the future about actually combining settings in our file from both builtin commands, and the UI (which I'm sure will be more flexible than a card), but yolo for now.

I'm merging this in so that you don't have to be the one to deal with my spec changes


chore(I18n): translation process updates

Summary by CodeRabbit

New Features

Introduced a new rule for JSON validation in localization files to ensure valid structure.
Enhanced internationalization (i18n) documentation with clearer guidelines and an overview of integration with translation management tools.

Bug Fixes

Improved ESLint configuration for better reporting of identical keys and ignored keys.

Documentation

Restructured the README for better clarity on internationalization processes, including an expanded implementation guide and a new section on automated workflows.
Updated project configuration details and ESLint validation instructions.

Chores

Updated dependencies to the latest version for improved functionality and performance.
Enhanced GitHub workflow configuration for better coverage reporting.
Improved Codecov configuration for more detailed coverage reporting and thresholds.

@coderabbitai ignore


feature: add security/web application firewall middleware

Feature: add security/WAF middleware
What did you do?
We use a web application firewall (www.signalsciences.com) to inspect and reject suspect we traffic. We are using traefik as an ingress controller for kubernetes. If we could add the signal sciences middleware to traefik, the WAF could protect all traefik traffic. To implement this without these code changes, you must run a nginx proxy with the signal sciences collector in reverse proxy mode wrapped around traefik. This is a lot of layers. By connecting the WAF middle ware directly I can avoid adding the reverse proxy and all the associated overhead.
What did you expect to see?
I expected a standard pattern to add optional middleware.
What did you see instead?
I see ways to add metrics, logging, but not firewall middlewares.
Who could work on this?
I'd gladly see if I can prototype something and submit a PR if the team maintainers think this is worthwhile.

Hello @mtbdeano .
Many thanks for your interest in the project.
Your need seems to be the same as the issue #1336.
That's why I close this issue in favor of it.
Fell free to re-open it if necessary.


New Architecture Fabric Support

Hey Any Idea will you guys will be updating to new architecture ?

https://www.youtube.com/watch?v=m8gyy7JItmU https://www.youtube.com/watch?v=1WZjeESH4a0


📝[Docs]: Changing of all issue templates and PR template

Wanted to change the layout of templates and pr.
Assign me under GSSOC

@Priyaaa1 please assign me this issue.

@Priyaaa1 please assign me this issue.


On first login to a new CF, it asks for a space but none exist

On a fresh Cloud Foundry deployment, there are no usable organizations or spaces yet.
So the new "ops user" experience looks like:
$ gcf login -u admin -p eaa139af52343
API endpoint: http://api.<IP_ADDRESS>.xip.io
Warning: Insecure http API endpoint detected: secure https API endpoints are recommended

Authenticating...
OK

Select a space:

Space> test
FAILED
Error finding space test
Space test not found

I'm facing this issue, jsut did a fresh installation and now I can't login to it because there are no Orgs or Spaces. I don't know how to proceed, my output is exactly the same as above. What's the solution taken in this case?


Sync with live

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/Schema/msbuild-targets.md
:white_check_mark:Succeeded

docs/Tools/dotnet-Commands.md
:white_check_mark:Succeeded

docs/What-is-NuGet.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Error: Unrecognized chain name or RPC url: rinkeby

I got this error after upgrading to @thirdweb-dev/sdk to 3.4.2 and @thirdweb-dev/react to 2.9.7
`Server Error
Error: Unrecognized chain name or RPC url: rinkeby
This error happened while generating the page. Any console logs will be displayed in the terminal window.
Call Stack
Object.getProviderForNetwork
file:///Users/apple/Documents/GitHub/safrootics-app-react/node_modules/<EMAIL_ADDRESS>(1195:15)

<EMAIL_ADDRESS>(745:49)
Array.reduce

<EMAIL_ADDRESS>(744:30)
Object.useMemo
file:///Users/apple/Documents/GitHub/safrootics-app-react/node_modules/react-dom/cjs/react-dom-server.node.development.js (1582:19)
Object.useMemo
file:///Users/apple/Documents/GitHub/safrootics-app-react/node_modules/react/cjs/react.development.js (1532:21)
ThirdwebProvider
<EMAIL_ADDRESS>(743:28)
processChild
file:///Users/apple/Documents/GitHub/safrootics-app-react/node_modules/react-dom/cjs/react-dom-server.node.development.js (3353:14)
resolve
file:///Users/apple/Documents/GitHub/safrootics-app-react/node_modules/react-dom/cjs/react-dom-server.node.development.js (3270:5)
ReactDOMServerRenderer.render
file:///Users/apple/Documents/GitHub/safrootics-app-react/node_modules/react-dom/cjs/react-dom-server.node.development.js (3753:22)`

hello @talhaa99 , can you also update react to 3.4.2?
if you're using Rinkeby contracts, keep in mind the chain has been deprecated and it won't work anymore.

hello @talhaa99 , can you also update react to 3.4.2?
if you're using Rinkeby contracts, keep in mind the chain has been deprecated and it won't work anymore.

I encountered the same issue, it broke my app in both the development and production phases.
I was using react and SDK version 2.9 and using both Mumbai and Polygon contracts for different deployment phases.
Then I upgraded the @thirdweb-dev/react and @thirdweb-dev/sdkboth to version3.4.4`  with migrations and now it works fine.


Cannot toggle display, but can toggle visibility?

I spent a lot of time trying to search for answers on this.  I know html2Canvas cannot render a div that is set to display:none, so cant a person set the display to inline prior to calling onRendered?  I can do this with visibility, but with display I just get a blank image:
This works:
        $(function () {
            $("#downloadCoupon").click(function () {
                var coupon = document.getElementById("coupon");
                coupon.style.visibility = "visible";
                html2canvas($("#coupon"), {
                    onrendered: function (canvas) {

                        theCanvas = canvas;

                        canvas.toBlob(function (blob) {
                            saveAs(blob, "<%=strCompany%>-<%=strID%>.png");
                        });
                    }
                });
                coupon.style.visibility = "hidden";
            });
        });

But this doesn't???
        $(function () {
            $("#downloadCoupon").click(function () {
                var coupon = document.getElementById("coupon");
                coupon.style.display = "inline";
                html2canvas($("#coupon"), {
                    onrendered: function (canvas) {

                        theCanvas = canvas;

                        canvas.toBlob(function (blob) {
                            saveAs(blob, "<%=strCompany%>-<%=strID%>.png");
                        });
                    }
                });
                coupon.style.display = "none";
            });
        });

EDIT: If there is some conflict here, is there some other work-around to basically hide the div onscreen, but still render it with html2Canvas?

Im confused.  I can change the visibility and postion of the DIV prior to rendering with html2Cavas, but just not the display?  The code below works (sets the div way off the screen, moves it to static before rendering, and then moves it back off the screen after)

Test

    $(function () {
            $("#downloadCoupon").click(function () {
                var coupon = document.getElementById("coupon");
                coupon.style.position = "static";
                html2canvas($("#coupon"), {
                    onrendered: function (canvas) {

                        theCanvas = canvas;

                        canvas.toBlob(function (blob) {
                            saveAs(blob, "<%=strCompany%>-<%=strID%>.png");
                        });
                    }
                });

                coupon.style.position = "absolute";
                coupon.style.left = "-9999px;";
            });
        });


penumbra-ibc: ⏎  invert early return condition

this diff is best viewed ignoring whitespace; use -w, or the "hide whitespace" option in github (the gear icon in the "files changed" tab).
this inverts our early return, so that we do not need to wrap the whole function body within a conditional block. this is a pure refactor that should have no visible changes to behavior.

re, lint failing, that is happening upstream on main. see https://github.com/penumbra-zone/penumbra/actions/runs/7698514268/job/20977960857

lint was kindly fixed in #3692, i am going to merge this now!


Missing Field 'Access Token'

Running current release on Ubuntu - package download, tar extract. Userid & password exported as per instructions but the client failing with the following error:
[2022-11-21T18:00:36Z ERROR tado_exporter::tado::client] unable to authenticate: error decoding response body: missing field access_token at line 1 column 63

+1


Package maintenance

Landed in https://github.com/metarhia/metawatch/commit/6147268cced769a241cb396f9870e2287370c53d and https://github.com/metarhia/metawatch/commit/7388d4e896266692f2240e82598f1838345b9996


fix godoc for RepositoriesService.RawFileContent

As an aside, would it be more clear if RawFileContent was named Blob instead?

Thanks @0xmohit! And yes, I think we can change it to GetBlob. Do you want to add that to this PR?

Done.

LGTM! Thanks @0xmohit!


Pulumi resource types should be linked in the documentation

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Currently pulumi's documentation shows attribute types but they are linked to the resource only if they are in the same page (? guessing from the pages I have read so far). It makes it difficult to navigate through them. e.g. https://www.pulumi.com/registry/packages/eks/api-docs/cluster/ creationRoleProvider of type CreationRoleProviderArgs is linked but instanceRole of type pulumi.aws.iam.role is not (written as pulumiAwsiamRole).
Affected area/feature

Pulumi packages documentation

Same issue here: https://www.pulumi.com/registry/packages/eks/api-docs/managednodegroup/#launch_template_python

this is likely related to: https://github.com/pulumi/pulumi/issues/14675
This issue is probably the same issue that we are having when it comes to linking to types that are defined in another package (i.e. linking to the aws iam role page from the eks pacakge)


Add transaction to nested repository.

This will let repository to begin transaction during creating many objects.

This change is

@jgoclawski review please :)

LGTM

Comments from Reviewable

Reviewed 4 of 4 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Encoding issues when pulling the kernel

When I try to pull a Kernel: ftnext/kaggle-spzcolab-201901 from Kaggle Kernels using Kaggle API,
kaggle kernels pull ftnext/kaggle-spzcolab-201901 -p . -m
I saw this information when python version is 3.6.7:
'gbk' codec can't encode character '\u25b6' in position 704: illegal multibyte sequence
I saw this information when python version is 2.7.15:
'latin-1' codec can't encode characters in position 178-187: ordinal not in range(256)
Could this problem happeen because of the Asian characters it has? Do I have a method to solve this problem?
Thank you.

It still has problem in the Kaggle API 1.5.13
I use the windows cmd to download the output of my kaggle notebook, using kaggle kernels output rock3yu/bronze-mold-demo1
It indead work for about 100 pics, after that the expection shows 'gbk' codec can't encode character '\u257a' in position 3627: illegal multibyte sequence
How can I fix the problems?


Crash on iOS when starting the app.

Please fill in the following fields:
Unity editor version: 2018.4.17f1
Firebase Unity SDK version: 6.15.2
Source you installed the SDK (.unitypackage or Unity Package Manager): .unitypackage
Firebase plugins in use (Auth, Database, etc.): Auth, Database, Analytics, Crashlytics, Messaging
Additional SDKs you are using (Facebook, AdMob, etc.): Facebook, Google Play Services, Sign In With Apple, Adjust
Platform you are using the Unity editor on (Mac, Windows, or Linux): Windows and Mac
Platform you are targeting (iOS, Android, and/or desktop): iOS and Android
Scripting Runtime (Mono, and/or IL2CPP): IL2CPP
Please describe the issue here:
The app crashes on app launch/coming from background after a long time on iOS. Occasionally while using the app also. The frequency of this crash has risen significantly since we started using the Firebase realtime database for tracking concurrent account use on multiple devices.
Callstack on Crashlytics:
Crashed: IL2CPP Threadpool worker
0  cricket                        0x103e72a4c il2cpp::vm::Array::GetByteLength(Il2CppArray*) + 77 (Array.cpp:77)
1  cricket                        0x103e51128 il2cpp::icalls::mscorlib::System::Buffer::BlockCopyInternal(Il2CppArray*, int, Il2CppArray*, int, int) + 31 (Buffer.cpp:31)
2  cricket                        0x104f7d46c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
3  cricket                        0x104f81774 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
4  cricket                        0x1040c0194 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
5  cricket                        0x1040c0490 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
6  cricket                        0x104107aa0 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
7  cricket                        0x104e8a38c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
8  cricket                        0x104e82f98 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
9  cricket                        0x104e89df4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
10 cricket                        0x104e8f3c0 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
11 cricket                        0x104e8de44 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
12 cricket                        0x1047f6a2c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
13 cricket                        0x1047e1714 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
14 cricket                        0x104a84440 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
15 cricket                        0x104d81130 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
16 cricket                        0x104e82f98 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
17 cricket                        0x104d80f50 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
18 cricket                        0x104e8a598 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
19 cricket                        0x104e8f040 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
20 cricket                        0x1047fa348 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
21 cricket                        0x1047e3ae0 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
22 cricket                        0x104a7ecc0 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
23 cricket                        0x104d81130 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
24 cricket                        0x104e82f98 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
25 cricket                        0x104d80f50 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
26 cricket                        0x104e8a598 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
27 cricket                        0x104e8f040 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
28 cricket                        0x104e8de44 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
29 cricket                        0x1047fb960 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
30 cricket                        0x1047e44b4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
31 cricket                        0x104a7e9e4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
32 cricket                        0x104d81130 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
33 cricket                        0x104e82f98 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
34 cricket                        0x104d80f50 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
35 cricket                        0x104e8a598 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
36 cricket                        0x104e8f040 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
37 cricket                        0x104e8de44 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
38 cricket                        0x1047f91c4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
39 cricket                        0x1047e29e4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
40 cricket                        0x104a7e3c4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
41 cricket                        0x104d81130 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
42 cricket                        0x104e82f98 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
43 cricket                        0x104d80f50 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
44 cricket                        0x104e8a598 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
45 cricket                        0x104e8f040 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
46 cricket                        0x104e8de44 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
47 cricket                        0x1047f6a2c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
48 cricket                        0x1047e1714 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
49 cricket                        0x104a82fa4 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
50 cricket                        0x104d81130 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
51 cricket                        0x104e82f98 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
52 cricket                        0x104d80f50 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
53 cricket                        0x104e8a598 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
54 cricket                        0x104e8f040 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
55 cricket                        0x104e8de44 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
56 cricket                        0x1047f6a2c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
57 cricket                        0x104b1bce0 leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
58 cricket                        0x104e9941c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
59 cricket                        0x103098e98 RuntimeInvoker_FalseSByte_t9070AEA2966184235653CB9B4D33B149CDA831DF(void (*)(), MethodInfo const*, void*, void**) + 42332 (Il2CppInvokerTable.cpp:42332)
60 cricket                        0x103e6534c worker_thread(void*) + 252 (ThreadPoolWorkerThread.cpp:252)
61 cricket                        0x103e85268 il2cpp::vm::ThreadStart(void*) + 717 (Thread.cpp:717)
62 cricket                        0x103e662b8 il2cpp::os::Thread::RunWrapper(void*) + 174 (Thread.cpp:174)
63 cricket                        0x103e684f0 il2cpp::os::ThreadImpl::ThreadStartWrapper(void*) + 106 (ThreadImpl.cpp:106)
64 libsystem_pthread.dylib        0x182c23914 _pthread_start + 168
65 libsystem_pthread.dylib        0x182c2b9d4 thread_start + 8

Please answer the following, if applicable:
Have you been able to reproduce this issue with just the Firebase Unity quickstarts (this GitHub project)?
We have noticed the crashes but not entirely sure how to reproduce it properly.
What's the issue repro rate? (eg 100%, 1/5 etc)
15%

Hi all!
I've been reading through this bug, and I think the most useful/actionable thing you could do is to reproduce it in the quickstart (ex: the database quickstart). This way we can quickly reproduce it on our end, add some sort of unit/integration test for it, and verify that the issue's been resolved.
With that said, I have a few thoughts that could help.
First, I notice that you mentioned that it occurred rarely before integrating Firebase and increased in frequency afterwards. Second I noticed that one of your threads seemed to have some sort of recursive call to WriteBatchInternal::SetContents which doesn't seem realistic quickly looking at this call (in release mode with inlining, this interpretation is not super accurate, so I can't be certain that this is actually an issue). The second crash looks like a marshalling error between native and managed code (I can't say which direction offhand), and doesn't look particularly related to Firebase but again it's hard to really tell. With this in hand, I'm guessing but can't be 100% certain that you're accessing bad native memory somehow - perhaps either via a heap corruption or some sort of use-after-free issue.
Now with your game being C# (Unity) and assuming well behaved native libraries (not a given, but a good starting point), the easiest issue to probe is the "use after free" issue. If those all are good assumptions:
Firebase makes judicious use of Tasks and native C# events that sometimes behave poorly around Unity's memory model (I've written this article loosely covering some of the issues). So if you could, it would be useful if you audited your use of Firebase's various callbacks.

I would suggest locating any time you += an event handler and ensuring that you -= it in the OnDestroy. Note that as is covered in this bug, you should cache your DatabaseReference rather than retrieving it by name in your OnDestroy to successfully unregister an event listener.

If you use ContinueWith or ContinueWithAsync, check if this != null in the continuation. What may happen is that the underlying native object backing any UnityObject, Component, MonoBehaviour, &c may get cleaned up between you firing a long running task and the continuation resuming. This could be exacerbated by the app getting suspended/resumed (ex: if you switch scenes or destroy a prefab when you return). Alternatively, convert your asynchronous continuations into Unity Coroutines since the Coroutine will not return from a yield after Destroy is called on its owning object. I like to use yield return new WaitUntil(()=>task.IsComplete); to do this.

If you use async/await, similarly check this != null after your yield statement. Again the underlying native object may go away.

Some people want to call Dispose on Firebase objects. We don't mark them as IDisposable to discourage this behaviour, and you should let Firebase manage this on its own (ie: don't call Dispose explicitly). If you're doing something along these lines in OnDestroy or OnApplicationPaused, perhaps it would trigger a use after free issue of some kind.

With this debugging, if you're still running into issues, perhaps you can share code snippets around where you're using Firebase so we can see if there are any issues. But really the best thing you can do to help us is reproduce it in the quick starts in this repository. Since they're well known and controlled, it's the easiest path to verifying your issue then verifying our fix.
Let us know what you find!

Hi all!
I've been reading through this bug, and I think the most useful/actionable thing you could do is to reproduce it in the quickstart (ex: the database quickstart). This way we can quickly reproduce it on our end, add some sort of unit/integration test for it, and verify that the issue's been resolved.
With that said, I have a few thoughts that could help.
First, I notice that you mentioned that it occurred rarely before integrating Firebase and increased in frequency afterwards. Second I noticed that one of your threads seemed to have some sort of recursive call to WriteBatchInternal::SetContents which doesn't seem realistic quickly looking at this call (in release mode with inlining, this interpretation is not super accurate, so I can't be certain that this is actually an issue). The second crash looks like a marshalling error between native and managed code (I can't say which direction offhand), and doesn't look particularly related to Firebase but again it's hard to really tell. With this in hand, I'm guessing but can't be 100% certain that you're accessing bad native memory somehow - perhaps either via a heap corruption or some sort of use-after-free issue.
Now with your game being C# (Unity) and assuming well behaved native libraries (not a given, but a good starting point), the easiest issue to probe is the "use after free" issue. If those all are good assumptions:
Firebase makes judicious use of Tasks and native C# events that sometimes behave poorly around Unity's memory model (I've written this article loosely covering some of the issues). So if you could, it would be useful if you audited your use of Firebase's various callbacks.
* I would suggest locating any time you `+=` an event handler and ensuring that you `-=` it in the `OnDestroy`. Note that as is covered in [this bug](https://github.com/firebase/quickstart-unity/issues/582), you should cache your [`DatabaseReference`](https://firebase.google.com/docs/reference/unity/class/firebase/database/database-reference) rather than retrieving it by name in your `OnDestroy` to successfully unregister an event listener.

* If you use `ContinueWith` or `ContinueWithAsync`, check if `this != null` in the continuation. What may happen is that the underlying native object backing any `UnityObject`, `Component`, `MonoBehaviour`, &c may get cleaned up between you firing a long running task and the continuation resuming. This could be exacerbated by the app getting suspended/resumed (ex: if you switch scenes or destroy a prefab when you return). Alternatively, convert your asynchronous continuations into Unity Coroutines since the Coroutine will not return from a yield after `Destroy` is called on its owning object. I like to use `yield return new WaitUntil(()=>task.IsComplete);` to do this.

* If you use `async/await`, similarly check `this != null` after your `yield` statement. Again the underlying native object may go away.

* Some people want to call `Dispose` on Firebase objects. We don't mark them as `IDisposable` to discourage this behaviour, and you should let Firebase manage this on its own (ie: don't call `Dispose` explicitly). If you're doing something along these lines in `OnDestroy` or `OnApplicationPaused`, perhaps it would trigger a use after free issue of some kind.

With this debugging, if you're still running into issues, perhaps you can share code snippets around where you're using Firebase so we can see if there are any issues. But really the best thing you can do to help us is reproduce it in the quick starts in this repository. Since they're well known and controlled, it's the easiest path to verifying your issue then verifying our fix.
Let us know what you find!

Thank you for your suggestions, we will definitely try these out and let you know.

Hi @ap4996. I believe the stacktrace symbolication is corrupted because of #673 or similar issue. Notice the same huge offsets: 58 cricket 0x104e9941c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
Crashlytics is missing symbols info so it takes the best guess, which is obviously wrong. The stacktrace at Unity Dashboard shows the same line with correct symbolication: 0x1041a9248 ThreadPoolWorkQueue_Dispatch_mCDF7415E4C9D02B34761CAE8EA15CD33DDF85ADF (Bulk_mscorlib_21.cpp:30760).

Hi @ap4996. I believe the stacktrace symbolication is corrupted because of #673 or similar issue. Notice the same huge offsets: 58 cricket 0x104e9941c leveldb::WriteBatchInternal::SetContents(leveldb::WriteBatch*, leveldb::Slice const&) (.cold.1) +<PHONE_NUMBER>
Crashlytics is missing symbols info so it takes the best guess, which is obviously wrong. The stacktrace at Unity Dashboard shows the same line with correct symbolication: 0x1041a9248 ThreadPoolWorkQueue_Dispatch_mCDF7415E4C9D02B34761CAE8EA15CD33DDF85ADF (Bulk_mscorlib_21.cpp:30760).

Can you elaborate on what that means? What do we make out of it? Out of memory or something?

@ap4996 Just that the stacktrace symbolication in crashlytics may be not correct. So you should look into symbolication provided by unity dashboard in this particular case. Meaning there's no problem with "leveldb" and there are no recursive calls of "WriteBatchInternal::SetContents".
As for the crash itself I think you should look into 0x0000000103417360 U3CU3Ec__DisplayClass174_0_U3CstartReceivingU3Eb__2_m014F19B4E9843E9BC00F7F0043B1EDC6E579AE2F (Bulk_Assembly-CSharp-firstpass_5.cpp:4506). This seems to be the only userspace part of stack trace. The name doesn't mean much to me, so I would probably look at line 4506 of this cpp file and try to understand what C# code get transpiled into this method. Hope this helps.

@L4fter oh ok..got it. Thanks for this. The line pointed to the websocket connection that we do. Will check it out if it fixes the issue.

@L4fter  Thank you very much for helping spotting the userspace part of the stack trace.
@ap4996
This may be related to the Unity bug described here https://github.com/firebase/firebase-ios-sdk/issues/5463#issuecomment-621445302, and this Github issue https://github.com/firebase/quickstart-unity/issues/781#issuecomment-707499969.
Since iOS team updated the SDK to avoid this Unity issue, it is worth it updating Firebase Unity SDK to 6.16.0, which includes this change.
Please let us know if this helps.
Shawn


key generate prb

i got the some errors,
Generating new keypair for eu-central-1.
Traceback (most recent call last):
  File "./scripts/keygen.py", line 11, in <module>
    conn_region = boto.ec2.connect_to_region(region)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/ec2/__init__.py", line 66, in connect_to_region
    return region.connect(**kw_params)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/regioninfo.py", line 187, in connect
    return self.connection_cls(region=self, **kw_params)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/ec2/connection.py", line 103, in __init__
    profile_name=profile_name)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/connection.py", line 1100, in __init__
    provider=provider)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/connection.py", line 569, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/auth.py", line 997, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV4Handler'] Check your credentials
Key already exists in AWS

what could it be meaning ?

It looks like you have a key already in that region. Try autovpn -D -r eu-central-1 -k eu-central-1_vpnkey

the error continues..
./autovpn -D -r eu-central-1 -k eu-central-1_vpnkey
WARNING: Keypair eu-central-1_vpnkey in eu-central-1 will be deleted. This is not reversible
Are you sure you want to continue? y/n :yTraceback (most recent call last):
  File "./scripts/delete_key.py", line 10, in <module>
    conn_region = boto.ec2.connect_to_region(region)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/ec2/__init__.py", line 66, in connect_to_region
    return region.connect(**kw_params)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/regioninfo.py", line 187, in connect
    return self.connection_cls(region=self, **kw_params)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/ec2/connection.py", line 103, in __init__
    profile_name=profile_name)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/connection.py", line 1100, in __init__
    provider=provider)
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/connection.py", line 569, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/usr/local/lib/python2.7/site-packages/boto-2.45.0-py2.7.egg/boto/auth.py", line 997, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV4Handler'] Check your credentials

Failed to delete keypair

Did you look at the aws management console for that region. Dose a key actually exists.

i checked it before commenting but it isn't seem still.

Taking a closer look it looks like your either missing credentials file or there is a typo in file. Error message shows "Check your credentials"


feat: support custom names for generated k8s resources

Relates to https://github.com/envoyproxy/gateway/pull/3527

hey @cnvergence can you elaborate why thats needed, the above diff is changing the value right before setting the resource name ?

oh @cnvergence I see the issue now, we are fetching based on name and not labels 😢
https://github.com/envoyproxy/gateway/blob/cdd4e065d4a2a5e3657052664073972f06ac77dd/internal/provider/kubernetes/predicates.go#L467
&
https://github.com/envoyproxy/gateway/blob/cdd4e065d4a2a5e3657052664073972f06ac77dd/internal/provider/kubernetes/predicates.go#L483
cc @dprotaso this is the issue seen in https://github.com/knative-extensions/net-gateway-api/pull/738

If this is addressed here, will it also fix when I rename the envoyDeployment:, the HPA envoyHpa? It no longer works it keeps the original generated name. There doesn't seem to be a way to patch that either

If this is addressed here, will it also fix when I rename the envoyDeployment:, the HPA envoyHpa? It no longer works it keeps the original generated name. There doesn't seem to be a way to patch that either

thanks for catching this ! ive fixed the deployment targetRef naming in hpa.
I havent changed the name on the hpa resource itself, that needs an API change, which can be handled in a future PR

On this same issue. I've renamed the deployment by patching the name, this causes the gateway to break. While the deployment with 3 replicas I get this error
  Last Transition Time:  2024-06-14T15:02:58Z
    Message:               The Gateway has been scheduled by Envoy Gateway
    Observed Generation:   1
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Last Transition Time:  2024-06-14T15:03:07Z
    Message:               Deployment replicas unavailable
    Observed Generation:   1
    Reason:                NoResources
    Status:                False
    Type:                  Programmed

the gateway status is this
NAME            CLASS           ADDRESS         PROGRAMMED   AGE
acme-envoy   acme-envoy   IPADDRESS   False        4m19s

@davem-git can you raise a separate GH issue to highlight the issue you are facing


Update _client.py

Suppressing tons of meaningless warning line when sending messages
solving issues like this
https://github.com/carpedm20/fbchat/issues/447

Closing, see https://github.com/carpedm20/fbchat/issues/447#issuecomment-524061613 for reasoning


Child Route - Cannot read property 'path' error

https://github.com/angular/router/tree/master/examples/angular-1/hello
the above example is supposed to demonstrate how a child route works, but it doens't appear to work in v0.5.  I keep getting the below error:
TypeError: Cannot read property 'path' of undefined
function serializeInstruction(instruction) {
    return JSON.stringify({
     // vvv error references this line
      path: instruction.path,
     // ^^^ error references this line
      component: instruction.component,
      params: Object.keys(instruction.params).reduce(function (acc, key) {
        return (key !== 'childRoute' && (acc[key] = instruction.params[key])), acc;
      }, {})
    });
  }

:+1:

:+1:

:+1:

I've already slated this for a release and confirmed the bug. No more +1's are necessary :)

Alright, thank you! This was actually my first Github comment, will hold back my +1's in the future to issues not confirmed as a bug and slated for a release ;-)

@leschirmeur – no worries.

will hold back my +1's in the future to issues not confirmed as a bug and slated for a release ;-)

Hah, sounds good. Cheers!

I have the same bug on 0.5.3. I need to know if it will be fixed soon : I'm choosing technos for a new project, and I'd like very much to use angular/ES6/new router, but if I can't use child routes, I wouldn't :(

Hi, thanks for Gerard this is a working plunk with child path and the new router:
Parent
In AppController.js is this:
$router.config([
    { path: '/home', component: 'home' },
    { path: '/users', component: 'users' },
    { path: '/', redirectTo: '/home' }
  ]);

Child
And UserController.js key part is this:
Controller.$routeConfig = [
  { path: '/:id', component: 'details' }
];

So, I just have to use $router.config instead of Controller.$routeConfig to make child route working ?

I have this error now : "Uncaught TypeError: Cannot read property 'canonicalUrl' of undefined".
But before looking forward, I just saw that when I'm using sibling views all views are reloaded when I change route, even views which keep the same component. Is there a way to avoid that ?

Really strange honestly. I guess Controller.$routeConfig should be used based on the plunk. I just use simple routes not child once currently.

Hello,
The bug is still there and what the Gerard's plunk demonstrates does not solve the issue but is mostly a dirty workaround. In a real case scenario you would not lead the user to a details/undefined route witch is the case in the plunk when it uses the ng-link="details({})" for the users button. You would certainly want a ng-link="users" instead, this raises the bug mentioned in this thread. You may think everything works fine because there are no url changes visible in the plunk editor, download the zip and run it locally.
Finally I wrote a basic hello world snippet to see the issue but can not find out why it breaks
Hope it helps

I think in happened because the router couldn't find component's file.

Hi Guys,
any news on this ?  Just like others says I am choosing whether I should go with the router-ui or this new router, where in real life you will have nested views


docs: typo in networks.md [APE-1329]

fix docs typo
What I did

Typo in networks.md
How I did it

/
How to verify it

/
Checklist

/

[x] All changes are completed
[ ] New test cases have been added
[x] Documentation has been updated

type check fail is unrelated and documented here: https://github.com/ApeWorX/ape/issues/1632


Define test datasets for population genomics

We want multiple test datasets to implement and test the population genomics features. Otter will be one of them, as we can refer to a publication that studies the population structure. We need at least another one, preferably on a smaller genome so that we can test things quicker.

[ ] Find all the otter data on ENA / NCBI
[ ] Find smaller / more diverse datasets

I want to first try running sarek with --step variant_calling, which skips the mapping steps. I am able to download the aligned .cram files for the Otter Lutra lutra. Based the cram header, they used reference file UR:/lustre/scratch120/npg_repository/references/Lutra_lutra/GCA_902655055.2/all/fasta/GCA_902655055.2_mLutLut1.2_genomic.fa. I am not sure which reference file it might be. It sounds like unmasked genome based on the file name.
The following fasta file Lutra_lutra-GCA_902655055.2-chromosomes.tsv.gz is listed in https://ftp.ensembl.org/pub/rapid-release/species/Lutra_lutra/GCA_902655055.2/ensembl/genome.
The following fasta file GCA_902655055.2_mLutLut1.2_genomic.fna.gz is listed in https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/902/655/055/GCA_902655055.2_mLutLut1.2/

Hi @hangxue-wustl . Those alignments were done by our Core Sequencing Facilities. It's a standard process that we can request if the species already has a reference genome assembly.
I've checked the methods they use and they don't explicitly say whether the assembly is taken masked or unmasked but for human they say they use https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/ which are unmasked.
On the other hand the filename is the same as the NCBI FTP https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/902/655/055/GCA_902655055.2_mLutLut1.2/GCA_902655055.2_mLutLut1.2_genomic.fna.gz where the assemblies are actually masked (with windowmasker). 🤷🏼
Our readmapping pipeline uses the unmasked genome to get all the possible hits, but then the alignments go through samtools view with these options https://github.com/sanger-tol/variantcalling/blob/main/conf/modules.config#L86
to remove non-primary hits. Faculty here told me that we should only be using the primary alignments for variant-calling.


Add badges

Fixes (part of) issue #135
@alessiostalla Could you have a look as well?

@dslmeinte looks good to me


map/area with impress

Hello again,
It is weird.
the "map" tag is not really used currently, but not deprecated or discontinued : it seems that all the browsers support that element (http://www.w3schools.com/tags/tag_map.asp)
With impress, only firefox and IE work !
An idea to solve it?
Thanks a lot in advance !

Sorry, I made a mistake ;)
Resolved.


add integration test for lncalendar

this PR can be closed and the test should be added into ext_repo https://github.com/lnbits/lnbits-extensions/pull/283


Xrootd vector read has no timeout

In https://github.com/scikit-hep/uproot4/blob/7e3353cfdec91c340c7a0e26a31a695d2db3bcf1/uproot/source/xrootd.py#L341-L343 there is no timeout argument passed to vector_read. The signature supports it: https://xrootd.slac.stanford.edu/doc/python/xrootd-python/modules/client/file.html#XRootD.client.File.vector_read and the regular read does use it:
https://github.com/scikit-hep/uproot4/blob/7e3353cfdec91c340c7a0e26a31a695d2db3bcf1/uproot/source/xrootd.py#L152-L154

Since self in the first code block is a XRootDSource, which has a self._timeout, is there any downside to calling the equivalent of _xrd_timeout and passing it to vector_read?
_xrd_timeout is unfortunately not defined for XRootDSource, but XRootDResource, though it could become a free function:
https://github.com/scikit-hep/uproot4/blob/7e3353cfdec91c340c7a0e26a31a695d2db3bcf1/uproot/source/xrootd.py#L90-L94

I think it should be as simple as adding timeout=self._resource._xrd_timeout() to the call. I'll make a PR soon

If the resource has the same _timeout (i.e. the data is simply copied). I would think that it is.

The answer to your last question is yes it does just copy the timeout currently.


Try to fix the build

Revert "Update extension telemetry module" - This reverts commit 282f499c2203a0cfa284e7a8c209963dfc123bac.

This was not the problem. Abandoning.