Unable to create oggStream

member.guild.channels.cache.get(voiceChannel.id).members.forEach((tempMember) => {
                const filename = './recordings/${Date.now()}-${tempMember.id}.ogg';

                const opusStream = receiver.subscribe(tempMember.id, {
                    end: {
                      behavior: EndBehaviorType.AfterSilence,
                      duration: 100,
                    },
                  });

                //const rawAudio = opusStream.pipe(new opus.Decoder({ frameSize: 960, channels: 2, rate: 48000 }));

                const oggStream = new opus.OggLogicalBitstream({
                    opusHead: new opus.OpusHead({
                        channelCount: 2,
                        sampleRate: 48000,
                    }),
                    opusTags: new opus.OpusTags({
                        maxPackets: 10,
                    })
                });

                const out = fs.createWriteStream(filename);

                pipeline(opusStream, oggStream, out,  (err) => {
                    console.log(err);
                    if (err) {
                        console.warn('❌ Error recording file ${filename} - ${err.message}');
                    } else {
                        console.log('✅ Recorded ${filename}');
                    }
                });
            });
        }

TypeError: failed to downcast any to number
at crc (C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\node_modules\node-crc\lib\lib.js:11:17)
at OggLogicalBitstream.calculateCRC (C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\node_modules\prism-media\dist\ogg\OggLogicalBitstream.js:92:23)
at OggLogicalBitstream.writePage (C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\node_modules\prism-media\dist\ogg\OggLogicalBitstream.js:150:33)
at OggLogicalBitstream.writeHeaderPages (C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\node_modules\prism-media\dist\ogg\OggLogicalBitstream.js:74:18)
at new OggLogicalBitstream (C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\node_modules\prism-media\dist\opus\OggLogicalBitstream.js:16:14)
at C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\index.js:51:35
at Map.forEach ()
at start (C:\Users\Yvtq8\Desktop\SideProjects\BobbyMcLovin\index.js:37:70)

Any updates on this? Or some alternative so I can record in a voice channel and output it?

Any updates on this? Or some alternative so I can record in a voice channel and output it?

I did manage to find a way to successfully record and store voice recordings. However, I have no clue how it works.´
I first made a class called Decoder
`
class OpusDecodingStream extends Transform {
encoder
constructor(options, encoder) {
    super(options)
    this.encoder = encoder
}

_transform(data, encoding, callback) {
    this.push(this.encoder.decode(data))
    callback()
}

}
Then with this decoder I still have no clue how it works, I somehow parsed the audio and was able to store it.await entersState(connection, VoiceConnectionStatus.Ready, 20e3);
if (connection) {
const receiver = connection.receiver;
member.guild.channels.cache.get(voiceChannel.id).members.forEach((tempMember) => {
const username = findUsername(tempMember.id)
const filename = ./recordings/test_${Date.now()}_${username}.wav;
            const encoder = new OpusEncoder(16000, 1)
            const commandAudioStream = receiver.subscribe(tempMember.id, {
                end: {
                  behavior: EndBehaviorType.AfterSilence,
                  duration: 100,
                },
            })
            .pipe(new OpusDecodingStream({}, encoder))
            .pipe(new FileWriter(filename, {
                channels: 1,
                sampleRate: 16000
            }))
        });`

Hopes this helps, but I really would like to see a cleaner guide. Not everyone is an audio engineer;(


fix about me card beyond height limit

Pull Request Template
The iPhone SE encounters an issue where the content overflows beyond the interface. Due to the inability to simulate on the Mac platform (equipment issues), I observed similar problems with devices having close screen sizes. Therefore, I will use the Android small phone emulator as the standard for fixing this issue.
Briefly describe the changes and the related issue.
Related Issue
Link to the issue here: #(issue)
Type of Change

[V] Bug fix (fix)
[ ] New feature (feat)

Checklist

[V] Code follows project guidelines
[ ] Self-reviewed code

Additional Notes
Any additional information.
Reviewers
xue,d5

previous view


urllib2 + cloudflare-scrape

Hi, how can i use cloudflare-scrape and urllib2 together? And is it possible at all?

Please read the integration section at the bottom of the README.
It's fairly easy to do. Just not as easy as with Requests.


Add missing Kotlin tests

Description
This PR bumps the coverage of the Android native code to 94%.

Also, all the warnings reported by Android Studio for the test files were fixed (except for one that was silenced).

For example:

Testing

[X] This change adds test coverage for new/changed/fixed functionality

Checklist

[ ] I have added documentation for new/changed functionality in this PR or in auth0.com/docs
[ ] All active GitHub checks for tests, formatting, and security are passing
[ ] The correct base branch is being used, if not master

Codecov Report

Merging #135 (0cf476a) into first-availability (b38e776) will increase coverage by 3.52%.
The diff coverage is n/a.

@@                   Coverage Diff                    @@
##             first-availability     #135      +/-   ##
========================================================
+ Coverage                 93.31%   96.83%   +3.52%
- Complexity                   69       75       +6
========================================================
  Files                        51       51
  Lines                       793      791       -2
  Branches                     43       43
========================================================
+ Hits                        740      766      +26
+ Misses                       46       13      -33
- Partials                      7       12       +5

Flag
Coverage Δ

auth0_flutter
100.00% <ø> (ø)

auth0_flutter_android
96.52% <ø> (+6.03%)
:arrow_up:

auth0_flutter_platform_interface
96.66% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...auth0_flutter/Auth0FlutterAuthMethodCallHandler.kt
100.00% <ø> (+11.11%)
:arrow_up:

...tlin/com/auth0/auth0_flutter/Auth0FlutterPlugin.kt
70.58% <ø> (+70.58%)
:arrow_up:

...th0_flutter/CredentialsManagerMethodCallHandler.kt
94.73% <ø> (+5.26%)
:arrow_up:

...h0_flutter/Auth0FlutterWebAuthMethodCallHandler.kt
87.50% <0.00%> (+12.50%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b38e776...0cf476a. Read the comment docs.


Small fix login.gov > Login.gov

Federalist build error looks unrelated? Hopefully? Maybe push an empty commit to kick it off again?

2020-06-19 15:32:59 ERROR [run-federalist-script] Error: Cannot find module 'uswds-gulp/config/browsers'

@zachmargolis it looks like there was a change to the gulp build for USWDS in gulp-uswds that caused this issue: https://github.com/uswds/uswds-gulp/commit/37bb3ad97a20bcee22bca0f8b528bef1820c85f3
This seemed to fix the issue, but I am not sure how it will effect :all-the-things: 🤔


Constructor with Params not working

Hi!
The BertModelLayer constructor expects 0 params (only self):
TypeError: init() takes 1 positional argument but 2 were given
The example at the Readme uses BertModelLayer(BertModelLayer.Params(...)) for original bert params, but it's not working.

@negedng - thank you for catching this, I just fixed it in the README, it should be:
BertModelLayer(**BertModelLayer.Params(...))

i.e. passing kargs to the BertModelLayer constructor.


Longest Bitonic subsequence

Is your feature request related to a problem? Please describe.
Longest Bitonic subsequence
Describe the solution you'd like
Longest Bitonic subsequence  in java python and cpp
Do you want to work on it
I would like to implement this in C++,python,java
Please assign me this issue as part of GSSOC `21

Please assign this to me in C++.I am a GSSOC'21 participant

Can I take this issue in Python?

@aaadddiii do it in java

I would like to implement this in C++.
Please assign me this issue as part of GSSOC `21
@aaadddiii


Installation error

Hi!
I just tried to install pAC. I followed the instructions and cloned the project into /Library/Python/2.7/site-packages/pinchesAndCacti/. I had found sonLib (installed using pip) in /Library/Python/2.7/site-packages/sonLib/. It gives the error
pinchesAndCacti severin$ sudo make all
cd externalTools && make all
cd threeEdgeConnected && /Library/Developer/CommandLineTools/usr/bin/make all
make[2]: *** No rule to make target `../..//../sonLib/lib/sonLib.a', needed by `../..//../sonLib/lib/3EdgeConnected.a'.  Stop.
make[1]: *** [all.threeEdgeConnected] Error 2
make: *** [externalToolsM] Error 2

I tried to uninstall sonLib through pip, clone it into place and install it that way, but that just causes an error for the sonLib install (below) and does not resolve the issue. Any ideas what to do? I'm on a macOS 10.13.5. Thanks a lot!
sonLib severin$ sudo make all
cd externalTools && make all
cd cutest && make all
clang -std=c99  -O3 -g -Wall --pedantic -funroll-loops -DNDEBUG  -c CuTest.c
ar rc cuTest.a *.o
ranlib cuTest.a
rm -f *.o
mv cuTest.a ../../lib/
cp CuTest.h ../../lib/
cd quicktree_1.1 && make all
gcc -c -I./include -O3 -Wall -o ./obj/align.o ./src/align.c
make[2]: *** [obj/align.o] Segmentation fault: 11
make[1]: *** [quicktreeM] Error 2
make: *** [externalToolsP] Error 2

Hi! You're right that installing sonLib through pip won't get you the compiled C library. (sonLib is sort of a weird hybrid of completely separate C/Python libraries in the same repository--a bad design but it sort of grew organically that way.)
You're on the right track by trying to compile sonLib, but your C compiler is crashing while trying to compile it. That usually means there's a compiler bug that we're somehow triggering. Can you paste in the output of clang --version and gcc --version?

Yeah, I sense a problem here, see below:
$ clang --version
Apple LLVM version 9.1.0 (clang-9<IP_ADDRESS>)
Target: x86_64-apple-darwin17.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
$ gcc --version
Segmentation fault: 11

OK, if you pull sonLib and get this commit, it should compile everything with one compiler rather than hardcoding gcc.

Thanks, that worked much better. Installation seemed to run (no errors or warnings thrown for what I can see), but make test still produces some errors. I can successfully install pAC now but the installation of mafTools returns errors. I'll first post the errors associated with sonLib:
$ sudo make test
[...]
======================================================================
ERROR: testFastaReadWriteC (bioioTest.TestCase)
Tests consistency with C version of this function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/sonLib/bioioTest.py", line 185, in testFastaReadWriteC
    system(command)
  File "/Library/Python/2.7/site-packages/sonLib/bioio.py", line 185, in system
    raise RuntimeError("Command: %s exited with non-zero status %i" % (command, sts))
RuntimeError: Command: sonLib_fastaCTest /tmp/tmpc8BYYj /tmp/tmpPziwtQ exited with non-zero status 127

======================================================================
ERROR: testCigarReadWrite (cigarsTest.TestCase)
Tests the C code for reading and writing cigars against the python parser for cigars.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/sonLib/cigarsTest.py", line 57, in testCigarReadWrite
    system(command)
  File "/Library/Python/2.7/site-packages/sonLib/bioio.py", line 185, in system
    raise RuntimeError("Command: %s exited with non-zero status %i" % (command, sts))
RuntimeError: Command: sonLib_cigarTest /tmp/tmpbaTcUC False exited with non-zero status 127

======================================================================
ERROR: testSonLibKVTokyoCabinet (kvdbTest.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/sonLib/kvdbTest.py", line 18, in testSonLibKVTokyoCabinet
    system("sonLib_kvDatabaseTest --type=tokyocabinet")
  File "/Library/Python/2.7/site-packages/sonLib/bioio.py", line 185, in system
    raise RuntimeError("Command: %s exited with non-zero status %i" % (command, sts))
RuntimeError: Command: sonLib_kvDatabaseTest --type=tokyocabinet exited with non-zero status 127

======================================================================
ERROR: testSonLibCTests (__main__.TestCase)
Run m,ost the sonLib CuTests, fail if any of them fail.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "allTests.py", line 30, in testSonLibCTests
    system("sonLibTests %s" % getLogLevelString())
  File "/Library/Python/2.7/site-packages/sonLib/bioio.py", line 185, in system
    raise RuntimeError("Command: %s exited with non-zero status %i" % (command, sts))
RuntimeError: Command: sonLibTests CRITICAL exited with non-zero status 127

----------------------------------------------------------------------
Ran 19 tests in 1.956s

FAILED (errors=4)
make: *** [test] Error 4

I'm not sure if that causes why the mafTools installation. Here's the mafTools make error trace:
$ sudo make
cd lib && make all
gcc -O3 -c -std=c99 -O3 -Wextra -Wall -Werror -pedantic -I ../external/ -I ../inc/ common.c -o common.o.tmp
make[1]: *** [common.o] Segmentation fault: 11
make: *** [lib.all] Error 2

Thanks a lot for your help!

The sonLib make test error is because it (sort of stupidly) assumes the test binaries are in your PATH. So PATH=`pwd`/bin make test should hopefully work.
The mafTools error is because mafTools is trying to use your gcc installation again (and your gcc install seems to be broken). I could try to submit a PR to mafTools to switch to clang on macs, but I don't have direct commit access.
Maybe an easier workaround might be to try to redirect from the faulty gcc install to your correctly working clang install. The redirect would only be temporary. So something like this would hopefully work (from the mafTools directory):
ln -s `which clang` gcc
ln -s `which clang++` g++
PATH=`pwd`:$PATH sudo make

That creates a couple links in the local directory and the final line says to prefer those links over your actual gcc install for just the following command (sudo make).

Hi! Thanks for your help, much appreciated. Sorry, I just realized I should have closed this a while ago ... Cheers!


Help --------How to send uint8_t array which has const byte  values to firebase?

Hi, I'm using adafruit pn532 library (https://github.com/adafruit/Adafruit-PN532) to read nfc tags. Here it is reading the unique id into a uint8_t  array as const byte values. How can we send this data into the real time  database?
Thank you. Hope you and your loved ones are safe. Thank you.

You can cast the uint8_t array to const char array (c string) and pass this value to setString or pushString function.
uint8_t myArr[] = {,,,,,};
Firebase.setString(firebaseData, (const char*)myArr, "Path_to_your_node");

When reading it back, cast it back to uint8_t for each char.
Firebase.getString(firebaseData, "Path_to_your_node");

uint8_t *myArr2 = new uint8_t[firebaseData.stringData().length()];
for (size_t i = 0; i< firebaseData.stringData().length();i++){
  myArr2[i] = (uint8_t)firebaseData.stringData().c_str()[i];
}

Hi,  I tried it .
Firebase.setString(firebaseData, (const char*)uid, "test1");
Since the uint8_t array has byte values it is not working properly.If you see below you can see the result in firebase.

The same value comes in arduino serial monitor when I'm using Serial.print function - Serial.print(uid);  . When I'm individually printing the values in each position using a for loop as given below it is working out fine.
for (uint8_t i=0; i < 7; i++)
    {
      Serial.println(uid[i]);
    }

The answer for the above code in serial monitor is this;
119
153
148
53
0
0
0
I think that the issue is in converting byte to char or string .  When the number of digits in a position is above 1 it is not working. What can we do here?
I want the result in firebase like this;
11915314853000

It is because the string is a null-terminated char array. If any byte data in your byte array is zero (null char), the casted string will be truncated.
Binary data in Firebase RTDB is not supported. The library provides an indirect way to store these binary data as base64 encoded string with function setBlob, pushBlob and getBlob.
Please see this example for the usage.

Hi , thank you for the fast response. I have one query.  Is there a way where we can send data without  the prefix blob,base64, . It's coming like this in the rtdb.

and I wish to have just the encoded value alone. What can be done here?
Thanking you in advance.

Did you see the example?
Use getBob function.

I'm making an android app where a string query will be done. If it is just the encoded string then it is easy to do in the RTDB , right?

See this https://github.com/mobizt/Firebase-ESP32/issues/65#issuecomment-602629455

Thank you, but won't it make the app slower with more computations. I'm just worried about the computation time.

Do you know about base64 encode/decode algorithm?
It just the basic thing in the computer world.
It also use in email transactions. The attachment (any binary data) will be converted to base64 string.
It can implement fast in 8-bit MCU, low clock speed as MCS51 too.
Please learn how to encode and decode first.


<esc> to close input eaves window in insert mode

hi @cseickel I just ran into this issue. I try to hit <esc> to close the popup inputs. The popup was closed, the window is still in insert mode. Is this a bug? or we need to add a custom keymap to also switch to normal mode?
Originally posted by @sandangel in https://github.com/nvim-neo-tree/neo-tree.nvim/issues/21#issuecomment-1074662177

thanks for creating this issue. 😀😀

Of course, when I got to fix it, it just completely stops happening! It may be because I upgraded to a new nightly neovim. Do you still have this issue @sandangel ?


 Kg3 {-4.16/28 16s} h6 {+1.21/27 30s} 2. Kh4 {-3.15/24 6.7s} g6 {+1.23/29 29s}
3. g4 {-3.43/31 18s} f4 {+2.01/32 28s} 4. Nxf4 {-2.71/25 7.9s}
g5+ {+2.11/30 26s} 5. Kh5 {-2.99/32 15s} gxf4 {+2.23/31 25s}
6. Kxh6 {-3.73/32 48s} Ne5 {24s} 7. Kg7 {-3.80/33 40s} Kd6 {+2.18/30 23s}
8. Kf6 {-4.26/30 19s} Nd7+ {22s} 9. Kf7 {-4.37/32 19s} Nf3 {+2.42/28 21s}
10. Nb5+ {-3.78/23 7.8s} Kd5 {+2.18/27 20s} 11. Ke7 {-3.73/29 11s}
Nc5 {+2.41/30 19s} 12. Kf6 {-4.13/30 0s, Black makes an illegal move: 0000} 1-0

Mess's pondering is probably a bit bugged right now. From the logs, it seems like Mess received the ponderhit command extremely fast after the go ponder. It probably wasn't able to start a search before the ponderhit command was received. That said, this issue needs some investigation. Thank you for the information.

@tissatussa Could you provide the average nps which Mess was reporting during this tournament?

it was not a tournament, it was a single game .. i showed the tail log lines, it's all i have .. mess does not output extensive info lines, like the opponent, having nps data .. however, running another game (6m3s) from normal starting position, mess DOES output such info lines and so i see an nps around 700000 upto 1 milion ..

With 1 Mnps, the issue of the ponderhit command arriving too fast can be discarded. The main issue which I see is that Mess was not reporting info properly for some reason, which is weird since it does print an info string for each depth it completes. This suggests that there was some misconfiguration when setting up the problematic game, though I have no idea what. Please do report if this occurs again.

Please do report if this occurs again

OK, i will do that.
btw. i do not always use Ponder -- it's a checkbox in CuteChess called "Thinking on opponent's time", but not all engines support it. I remember that game had Ponder ON. Might the Ponder option have been the cause of the problem ? Or do you have no clue ?

btw. i do not always use Ponder -- it's a checkbox in CuteChess called "Thinking on opponent's time", but not all engines support it. I remember that game had Ponder ON. Might the Ponder option have been the cause of the problem ? Or do you have no clue ?

As I said:

Mess's pondering is probably a bit bugged right now.

So yeah, probably.


Determine path for serialization + widgets

Within NcEditor, each component type needs a way to expose its properties through the UI and be saved/loaded. We need to determine how game-side components will provide this functionality to the editor.
Q: How will the editor loop through each component type?
We'd like to be able to say something like:
for (auto& c : registry->ForEachComponent(entity))
{
    Serialize(stream, c);
}

This doesn't work for pooled components, since there's no common base class. We don't have any way to loop through any sort of 'type list' of all registered types - we just have a list of PerComponentStorageBase.

Could we make some common interface wrapper that exposes funcs like Serialize() and UIWidget()?
Could we manage a type list somehow and loop over that? The use would need to 'set' this somehow.

Q: What does (de)serialization interface look like?
Component constructors may need access to all kinds of things during deserialization + they need to be constructed through the registry.
Q: How do we handle logic components + other user logic? If we load a scene by deserialization, maybe there is some game function that should run after that process to cover something not possible during deserialization.

If we load a scene by deserialization, maybe there is some game function that should run after that process to cover something not possible during deserialization.

This crossed my mind as well - personally I broke up creating a scene into two parts. First step was defining entity/component/storage stuff as well as something like the "logic types" where you map a function definition to some type. The second setup function was to initialize the data when the scene was not being restored from files. Then there is an amount of validation that needs to be done (if the data in the scene depends on something that was removed that's not good. like if you had EdgePanCamera logic the last game run and then you removed it but there's a serialized entity that depended on it.
Just some unsolicited thoughts!

In a previous POC I had made it possible to mix source code + deserialization during scene initialization, but never got far enough to handle logic serialization. So you could do something like, but PostLoad() would always need to know how to setup logic components:
void MyScene::Load()
{
    Init();
    HasSaveData() ? Deserialize(GetSaveData()) : FirstTimeInit();
    PostLoad();
}

I've started with the first question, but I need to figure out what it would look like to add some sort of function registry to track logic types for serialization (and possibly selecting/assigning functions through the editor).


Fix wrong links in porch install documentation

was attempting to merge https://github.com/GoogleContainerTools/kpt/pull/3176 but mis-merged this PR.
I uploads the fix to resolve my comment https://github.com/GoogleContainerTools/kpt/pull/3177


TM related steps on Migration steps are missing

Description:
In the APIM Migration guide [1], there are no steps or any comment related to Traffic Manager node. If user can copy the custom throttling plans to new version or have to recreate in the new version please mention it.
[1] https://docs.wso2.com/display/AM260/Upgrading+from+the+Previous+Release
Affected Product Version:
2.6.0

@vithu30 , please share your comments on this git issue.

We will not be fixing this as this issue is based on an older version of the product.


#2 - Add Escape button functionality

Implemented close_on_escape
closes #2

Looks good! Thanks @ObieMunoz!

Happy to help!


Design Edits - Japanese Club

Japanese Club Cover:

[x] It's missing, please add if possible? :o (Mirooo)
[x] Scale borders so theres more room at the sides and tops

Interaction Pages:

[x] If possible, please change the blue choice buttons to the same color as text or use just a black button (Ik marie did that but it doesn't work here in my opinion) Open for thoughts otherwise though
[x] Scale borders so theres more room at the sides and tops

Bio Pages:

[x] Replace ugly wiggly line with the straight line in Teams folder pls 😺 emotes wow 🪱
[x] For bio pages only, make the header (names) centered underneath the portraits rather than on top
[x] Scale borders so theres more room at the sides and tops

Note: These are just some things that can be changed now, I'll just talk to you guys tomorrow about specifics

everything should be done now, we'll check with you tomorrow


How to run these JFileman??

I downloaded and don't know how to run these code bro.

Honestly it probably doesn't run.  It was something I was playing with several years ago as a replacement for an PHP based file manager we used internally at my job.  I never got time to work on it, and the need for it went away so it is just kind of hanging out basically dead.

ok, thanks Sid. for your reply.


NULL checks before free() unnecessary

free(NULL) is a no-op, so we can simplify code that currently looks like if (ptr != NULL) free(ptr); to free(ptr);
See e.g. https://stackoverflow.com/questions/1938735/does-freeptr-where-ptr-is-null-corrupt-memory

Good idea. I wasn't certain that was portable, but it is. This would be cleaner 👍

I like this.
Talking about malloc hygiene -- and sorta hjiacking this thread as well -- something cool that Tor does is that it has a tor_free() macro that sets the pointer to NULL after freeing it: https://github.com/torproject/tor/blob/main/src/lib/malloc/malloc.h#L34
This makes it much harder to double free things. I was thinking of porting this to c-kzg.


Add Notifications API

[x] reviewed prior work, or confirmed not already in progress
[x] moved files
[x] updated filename if needed (eg remove "TC39-" or s/webdriver2/webdriver/)
[x] removed draft date
[x] determined name
[x] added description (triangulated from mdn, spec, caniuse)
[x] added entry point
[x] verified baselines
[x] checked api list against mdn list
[x] added to a group or not
[x] determined whether the feature should be split
[x] looked for other related draft features that could be included
[ ] what else?

See also
#960 and #973

Wrote up the feature composition proposal here: https://github.com/web-platform-dx/web-features/issues/973#issuecomment-2360576590
We can consider a group once we get to push notifications.

I had a look at this and ended up poring over everything to do with notifications. It's very messy—primarily because caniuse describes this feature in a peculiar way—so we need to tread carefully as long as we're setting the caniuse key on this feature. See https://github.com/web-platform-dx/web-features/issues/973#issuecomment-2364007085 for a much, much more detailed take.

Re-requested review from @captainbrosset for the description shortening in notifications-apps.yml.

This now has the BCD changes from https://github.com/mdn/browser-compat-data/pull/24705 and implements the approach recommended in #973.


139 company identity setup

Draft to get some input on the current design.
This is the first step for implementing #139
It refactors file uploading to a file_upload_store and file_upload_service, so we can re-use that logic across multiple areas.
It adds the following infrastructure for Company identity:

Data models
Persistence
Service
Web endpoints

And a basic implementation of all endpoints I sketched out on Postman.
What's missing are tests and going through the different interactions to see if they are fine as they are API-wise.
Further, once this is merged, I will start working on the DHT-based logic, of how to propagate companies across signatories etc.

Yes, somewhat, especially when we implement the identity and company blockchains.
Anywhere I can read up on that?


Incorrect line counter

Description
Expected to see correct accordance of the lines in "Find Results" window and in "Disassembly".
In the parentheses you can see number 35, meanwhile this line is one line above the place, where the necessary instruction is written.
Steps to reproduce
In my capture.
0) Unzip capture.

Open a command with in Event browser with EID 296.
Go to Pipeline State window.
Choose vertex shader.
Click on "View" near the shader.
Press "Crtl" + "F" and write down "ld_indexable".
Find all.

I've sent a capture to the mail with the theme "Hardspace Shipbreaker.zip".
Environment

RenderDoc version: 1.11
Operating System: Windows 10
Graphics API: DirectX11

I've not tested it on all events, but I've seen the same problem everywhere, where I've tried to.

RenderDoc is only intended for capturing your own programs. Capturing copyrighted programs that you do not have the rights to is not endorsed and I will provide absolutely no support for it.

RenderDoc is only intended for capturing your own programs. Capturing copyrighted programs that you do not have the rights to is not endorsed and I will provide absolutely no support for it.

Opening any DXBC shader and using text search should probably be enough as reproduction steps.

Opening any DXBC shader and using text search should probably be enough as reproduction steps.

I finally understood what was incorrect.
I thought that :

check this on NDA project will change the steps. (it won't)
After closed first issue I thought it will be better to open a new one( bad idea)
Even in new issue I didn't give a minimal example to reproduce the bug.
I wasn't right. Sorry, Baldur Karlsson.

I finally understood what was incorrect.
I thought that :

check this on NDA project will change the steps. (it won't)
After closed first issue I thought it will be better to open a new one( bad idea)
Even in new issue I didn't give a minimal example to reproduce the bug.
I wasn't right. Sorry, Baldur Karlsson.


Add LogConfig and Ulimits(new in docker 1.6).

Implement the new feature in docker 1.6

Ulimits - A list of ulimits to be set in the container, specified as { "Name": , "Soft": , "Hard":  }, for example: Ulimits: { "Name": "nofile", "Soft": 1024, "Hard", 2048 }}
LogConfig - Logging configuration to container, format { "Type": "<driver_name>", "Config": {"key1": "val1"}} Available types:json-file,syslog,none.json-file` logging driver.
Add two types for HostConfig

ping @vieux @abronan

LGTM.
I tested by running ContainerCreate with
HostConfig: dockerclient.HostConfig{
                        Ulimits: []dockerclient.Ulimit{
                                {
                                        Name: "nofile",
                                        Soft: 1024,
                                        Hard: 2048,
                                },
                        },
                        LogConfig: dockerclient.LogConfig{
                                Type: "none",
                        },
                },

and the docker inspect for the created container had the expected Ulimits and LogConfig settings - the default seems to be "json-file" logging and no Ulimits.

ping @aluzzardi @ehazlett

LGTM


Web assembly error with @vercel/og

What version of Remix are you using?
1.8.0
Steps to Reproduce
Install @vercel/og add the following line to your remix.config.js: serverDependenciesToBundle: ['yoga-wasm-web', '@vercel/og', 'satori'] and run npm run dev.
Expected Behavior
Developer server runs and the @vercel/og package works.
Actual Behavior
Receive the following error:
/Users/leonard/Desktop/Development/vercel-og-wasm/node_modules/yoga-wasm-web/dist/index.js:1
import i from"./entry.js";import o from"./yoga.mjs";function s(e,t){return t}async function c(e){let t=await o({instantiateWasm(r,a){return WebAssembly.instantiate(e,r).then(n=>{a(n.instance||n)}),{}},locateFile(){return""}});return i(s,t)}async function l(e){let t=await o({instantiateWasm(r,a){return WebAssembly.instantiateStreaming(e,r).then(n=>{a(n.instance||n)}),{}},locateFile(){return""}});return i(s,t)}export{c as default,l as initStreaming};
                                                                                ^
TypeError: WebAssembly.instantiate(): Argument 0 must be a buffer source or a WebAssembly.Module object
    at Object.instantiateWasm (/Users/leonard/Desktop/Development/vercel-og-wasm/node_modules/yoga-wasm-web/dist/index.js:1:153)
    at /Users/leonard/Desktop/Development/vercel-og-wasm/node_modules/yoga-wasm-web/dist/yoga.mjs:65:421
    at /Users/leonard/Desktop/Development/vercel-og-wasm/node_modules/yoga-wasm-web/dist/yoga.mjs:66:356
    at c (/Users/leonard/Desktop/Development/vercel-og-wasm/node_modules/yoga-wasm-web/dist/index.js:1:110)
    at Object.<anonymous> (/Users/leonard/Desktop/Development/vercel-og-wasm/node_modules/@vercel/og/src/og.ts:17:25)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)

See the following repo to reproduce: https://github.com/benjaminleonard/vercel-og-wasm

@benjaminleonard Did you figure out a solution here? Can you use vercel edge functions here?

Their documentation explicitly says that only their edge runtime is supported.

Only the Edge Runtime is supported. The default Node.js runtime will not work. To use Node.js or any other runtime, you can use Satori directly.

I've been working on a Node.js-friendly alternative today and it's turning out to be decent. Right now I just want to make sure I'm not infringing any license before sharing the code but while I figure this out, please let me know if it's something you would be interested in using.

Now vercel supports the edge runtime for remix in production
I deployed a short example using @vercel/og to https://vercel-og-remix-fedeya.vercel.app
here is the repo: https://github.com/fedeya/vercel-og-remix.
The only problem i found right now is getting it to work locally, i get this error when i request that route in local
Unexpected Server Error

TypeError [ERR_INVALID_URL]: Invalid URL

@fedeya could you try using @m5r/og? It's the same API so it should be a drop-in replacement.
The package's repo is hosted here: https://github.com/m5r/og

Since this runs in production, and has a lot of moving parts, and is four months old, please let us know if there's anything Remix needs to do for this to work in dev too. Thanks for understanding.


FIX-#6632: Return Series instead of Dataframe for groupby.apply in case of experimental groupby

What do these changes do?

[x] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

[x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] signed commit with git commit -s
[x] Resolves #6632
[x] tests added and passing
[ ] module layout described at docs/development/architecture.rst is up-to-date

@dchigarev any comments?

@dchigarev any comments?

Well, the only thing to notice is that it now always triggers the computation after groupby (because of the .columns access).
Previously we had a trend of getting rid of such explicit materialization, and now we're adding them back again. The changes are introduced to the general groupby method ._wrap_aggregation(), meaning that all the aggregations will be affected by the changes, not only .apply() calls for which we have to infer a new return type.
If we decide to keep the fix, maybe we should at least move it to the .apply() method, so the other methods won't be affected?

script to measure
import modin.pandas as pd
from asv_bench.benchmarks.utils.common import execute
import modin.config as cfg
import numpy as np
from timeit import default_timer as timer

# initialize workers
pd.DataFrame([cfg.NPartitions.get() * cfg.MinPartitionSize.get()]).to_numpy()

df = pd.DataFrame({"by_col": np.tile(np.arange(10_000), 100), "a": np.arange(1_000_000), "b": np.arange(1_000_000), "c": np.arange(1_000_000)})
execute(df) # trigger import

results = []

t1 = timer()
for _ in range(10):
    results.append(df.groupby("by_col").sum())
[execute(df) for df in results]
print(timer() - t1)

t1 = timer()
res = df.groupby("by_col").sum()
print(timer() - t1)

@dchigarev any comments?

Well, the only thing to notice is that it now always triggers the computation after groupby (because of the .columns access).
Previously we had a trend of getting rid of such explicit materialization, and now we're adding them back again. The changes are introduced to the general groupby method ._wrap_aggregation(), meaning that all the aggregations will be affected by the changes, not only .apply() calls for which we have to infer a new return type.
If we decide to keep the fix, maybe we should at least move it to the .apply() method, so the other methods won't be affected?

script to measure
import modin.pandas as pd
from asv_bench.benchmarks.utils.common import execute
import modin.config as cfg
import numpy as np
from timeit import default_timer as timer

# initialize workers
pd.DataFrame([cfg.NPartitions.get() * cfg.MinPartitionSize.get()]).to_numpy()

df = pd.DataFrame({"by_col": np.tile(np.arange(10_000), 100), "a": np.arange(1_000_000), "b": np.arange(1_000_000), "c": np.arange(1_000_000)})
execute(df) # trigger import

results = []

t1 = timer()
for _ in range(10):
    results.append(df.groupby("by_col").sum())
[execute(df) for df in results]
print(timer() - t1)

t1 = timer()
res = df.groupby("by_col").sum()
print(timer() - t1)

@dchigarev Good catch!
I was surprised that even df.groupby("by_col").sum() does not calculate the columns in advance. It doesn't look that difficult.

If we decide to keep the fix, maybe we should at least move it to the .apply() method, so the other methods won't be affected?

At least yes, you are right.

If we decide to keep the fix, maybe we should at least move it to the .apply() method, so the other methods won't be affected?

you mean here - https://github.com/modin-project/modin/blob/master/modin/pandas/groupby.py#L657?

you mean here - https://github.com/modin-project/modin/blob/master/modin/pandas/groupby.py#L657?

yes

you mean here - https://github.com/modin-project/modin/blob/master/modin/pandas/groupby.py#L657?

yes

Okay, pls look at another attempt


HDDS-2531. Sonar : remove duplicate string literals in BlockOutputStream

What changes were proposed in this pull request?
Refactored code to fix issues prompted by Sonar.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2531
How was this patch tested?
Syntax changes and logging format changes.

This jira seems to be inactive for a while. Does the patch need updating?

This jira seems to be inactive for a while. Does the patch need updating?

One minor edit needed: add if (LOG.isDebugEnabled())

/pending "One minor edit needed: add if (LOG.isDebugEnabled())"

/close

/debug

/close


[SPIKE] Research ELK/OpenSearch policy management for its default indices

Is your spike related to a problem or idea? Please describe.
By default ELK stack doesn't have any policy management for its default indices. There are security indices that are producing data all the time which ends with taking over all possible shards on ElasticSearch and ELK stack stops working.
Example issue from a customer:
{"agent":{"ephemeral_id":"cfcd68d5-ec76-424e-b2eb-1e89d1ac57c3","hostname":"lahcmonitor01d","id":"474b78b4-60ff-4fc5-9147-acce120f1dd2","name":"lahcmonitor01d","type":"filebeat","version":"7.9.2"},"ecs":{"version":"1.5.0"},"host":{"name":"lahcmonitor01d"},"input":{"type":"log"},"log":{"file":{"path":"/var/log/messages"},"offset":703865869},"message":"Feb 21 09:05:19 lahcmonitor01d performance-analyzer-agent-cli: 09:05:19.140 [pa-reader] ERROR com.amazon.opendistro.elasticsearch.performanceanalyzer.reader.ReaderMetricsProcessor - Error reading file '/usr/share/elasticsearch/data/batch_metrics_enabled.conf': java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/batch_metrics_enabled.conf"}, Private:file.State{Id:"native::3195-64773", PrevId:"", Finished:false, Fileinfo:(*os.fileStat)(0xc000c24680), Source:"/var/log/messages", Offset:703866222, Timestamp:time.Time{wall:0xc07d018e2a7d621c, ext:8371919972062144, loc:(*time.Location)(0x419f7a0)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0xc7b, Device:0xfd05}, IdentifierName:"native"}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"validation_exception","reason":"Validation Failed: 1: this action would add [6] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}
This indices are created by default (tbh same as filebeat*) so there should be some default policy to prevent this situation from happening.
Describe the outcome you'd like
Check what kind of Policy management we can offer to keep this issue from happening in the form of additional issues. Note that these (might/should) need to cover the following:

Elasticsearch for version 1.0.3
Opensearch for Epiphany 2.0.1

What is the reason or source for the spike
Internal team
Additional context
None

DoD checklist

[ ] Reader is able to understand the results of spike
[ ] The results of the spike are presented in a table (to show simply what are compared or researched parameters) / not applicable
[ ] Each value / cell in the results table is described more deeply below
[ ] Demo of the spike (automated as much as possible)
[ ] Design doc updated

We have quite a good and short documentation with a policy example - https://github.com/epiphany-platform/epiphany/blob/develop/docs/home/howto/LOGGING.md#how-to-manage-opendistro-for-elasticsearch-data
In fact this policy should depend on the size of machines storage and their number, which is configurable. So we could disable an audit logging by default, or make this option configurable:

https://github.com/epiphany-platform/epiphany/blob/0f0e01a784afc00931116a13a1a638ba6a72e9d8/ansible/playbooks/roles/opendistro_for_elasticsearch/templates/elasticsearch.yml.j2#L135
https://opendistro.github.io/for-elasticsearch-docs/docs/security/audit-logs/

Or configure some policy, but we can't be sure in the requirement prevent this situation from happening - this should be analysed by the user, and taking into account the disk size, based on their experience, configured.
Security/audit can be disabled/configured via Kibana UI, the same with policies:

According to documentation it's almost the same for opensearch:

https://opensearch.org/docs/latest/security-plugin/audit-logs/index/
https://opensearch.org/docs/latest/im-plugin/ism/index/

As I see in PR #3093, we are going to enable audit logging by default again:

If PR changes are accepted, similar should be done for v2.0.1 after Opensearch migration.

❗ ❗ ❗ Similar work should be done for v2.0.1 after Opensearch migration.

@rafzei Is this something we can include in the current OpenSearch PR maybe?

This PR is quite big already. I would recommend doing it in the separated one.

This PR is quite big already. I would recommend doing it in the separated one.

Could you create a new issue for that for 2.0.1, linking back to this issue and blocked by your opensearch one?

The OpenSearch part is deferred to #3140, we can close this once tested for OpenDistro.


ex11_9_10.cpp can't be successfully compiled !

Exactly!
Because ex11.10 is trying to show an illegal syntax. The following is the description:
//! Exercise 11.10:
//! Could we define a map from vector<int>::iterator
//! to int? What about from list<int>::iterator to int?
//! In each case, if not, why not?

OK , I got it ! I ignored it in my submission!

Cool.


The version of quic-go you're using can't be built on Go 1.19 yet

$ go install github.com/neex/http2smugl@latest
go: downloading github.com/neex/http2smugl v0.0.0-20220707215445-78abc0948765
go: downloading github.com/lucas-clemente/quic-go v0.27.2
go: downloading github.com/marten-seemann/qpack v0.2.1
go: downloading github.com/spf13/cobra v1.4.0
go: downloading golang.org/x/net v0.0.0-20220617184016-355a448f1bc9
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/cheekybits/genny v1.0.0
go: downloading golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading github.com/marten-seemann/qtls-go1-18 v0.1.2
# github.com/lucas-clemente/quic-go/internal/qtls
go/pkg/mod/github.com/lucas-clemente/quic-go@v0.27.2/internal/qtls/go119.go:6:13: cannot use "The version of quic-go you're using can't be built on Go 1.19 yet. For more details, please see https://github.com/lucas-clemente/quic-go/wiki/quic-go-and-Go-versions." (untyped string constant "The version of quic-go you're using can't be built on Go 1.19 yet. F...) as int value in variable declaration
$ go version
go version go1.19.3 linux/amd64

Hi!
That should be fixed now.

Hi @neex
I have this error with go1.20
Can you build the code and add it to Releases


Bump pygmt from v0.2.0 to v0.2.1

@seisman, not sure what's up with the repo2docker Validator, but have a go at it in the morning when you're awake.

@seisman, not sure what's up with the repo2docker Validator, but have a go at it in the morning when you're awake.

Perhaps because you're using your own fork?


Add default model settings

Default model, scroll on chat sent/received, continue conversations on reopen, hide menu bar, also gotta add some of those features though.


MGMT-11688: enable bumping 4.11 RHCOS

Official RHCOS images for 4.11 have been created, so we can remove "4.11" from skip-list.

/test all

/cc @danielerez


Consider having both "substr" and "substring" functions in Calc

JavaScript has substr(start [, length]) and substring(start [, end]) forms.
Currently Calc has substr(string, start [, end]) which is kind of not either one, and Java has substring(start [, end]) which is at least consistent with JavaScript. Also, JavaScript substr accepts negative start, while substring does not.  Java's substring does not accept negative indices.  Calc's substr accepts either or both negative...
So, we currently have an interestingly paradoxical situation.

Well, we have implemented "slice" notation, and updated "substr" to use start, length. So, I'm going to close this as already solved by these other changes.


Loading only encoder part of a pretrained enc-dec transformer model in fairseq

❓ Questions and Help
I am new to fairseq and wanted to ask you how I can load an encoder part so that I can use it in another model.
from fairseq.models.transformer import TransformerModel
en2de = TransformerModel.from_pretrained("checkpoint/wmt16.en-de.joined-dict.transformer",
checkpoint_file='model.pt',
data_name_or_path='checkpoint/wmt16.en-de.joined-dict.transformer',
bpe='subword_nmt',
bpe_codes='checkpoint/wmt16.en-de.joined-dict.transformer/bpecodes')

Hey @semerekiros perhaps you can load the entire model before extracting only the encoder?
I think it would more more difficult to just load the encoder since fairseq saves model states (namespaces play a key part and there will be complexity involved when renaming them to only the portions involving the encoder).

Yes, I would follow @mingruimingrui's advice.

Solved it. Thanks


Android GHA workflow

Add Android as a GHA CI worfklow.

added android.yml to our workflows
fixed an analytics build issue which now news to include play-services-base as an implementation dependency.
there's a lint failure in a firestore dependency.  Disabling abortOnError for that gradle build target for now. I've added an internal issue to the backlog to circle back on this issue later.

Looks good, though just a heads up I may change the way it decides whether to install ccache as part of my change. :)

That's ok!


User always required to login after 20 minutes

I'm hoping someone can point me in the right direction with this.
I can't seem, no matter what I try, to increase the time required between user logins more than 20 ~ 30 minutes. This is even if the user is continuously active - so this is not "idle" time it's maximum time between logins. After 20 minutes the user is redirected to the login page to login again.
The strange thing is that when I decrease the CookieOptions.ExpireTimeSpan to something like 10 seconds, it redirects after 10 seconds as expected. I also tested the SlidingExpiration and that works too with the very low ExpireTimeSpan value ( the user keeps working till the total time spent is more than 20 minutes after which it redirects again - as if 20 minutes was the MAX you can ever go without logging in). When I set the ExpireTimeSpan larger than 20/30 minutes doesn't seem to work and it forces me to login again.
I thought this might have something to do with my host server's application pool settings. Maybe the application pool dies after 20 minutes causing the login to be invalid but it happens even when I run the application straight from Visual Studio using IISExpress.
The problem is I don't even know what to try or where to look. I've completely run out of ideas  and I don't know how to debug this if this is a cookie issue.
To Reproduce
This is an ASP.NET Core 2.1 application.
I set the ExpireTimeSpan to 1 hour and the SlidingExpiry to true. Once I log in I set a stop watch to 15 minutes. At 15 minutes I navigate to one of the pages. After another 15 minutes I navigate to another page but it always redirects me to the login page at this point. I tried this with different scenarios - one time navigating pages every 4 or 5 minutes.. it always redirects to the login page at around 20 ~ 30 minutes (approximate since I lose count sometimes and it takes time to go to the screen and click and then reset my countdown times)
This is my code currently on Startup:
  public void ConfigureServices(IServiceCollection services)
  {
    services.Configure<CookiePolicyOptions>(options =>
    {
        // This lambda determines whether user consent for non-essential cookies is needed for a given request.
        options.CheckConsentNeeded = context => false;
        options.MinimumSameSitePolicy = SameSiteMode.None;
    });
     //services.AddSingletons, Transients etc..
    services.AddIdentity<IdentityUser, IdentityRole>()
          .AddDefaultTokenProviders();

    services.Configure<IdentityOptions>(options =>
    {
        // Password settings.
        // Lockout settings.
        // User settings.
    });

    services.ConfigureApplicationCookie(options =>
    {
        options.LoginPath = "/Login";
        //sliding expiration doesn't seem to work, nor does expiretimespan
        options.SlidingExpiration = true;
        options.ExpireTimeSpan = TimeSpan.FromHours(1);
    });

    services.AddMvc(config =>
    {
        var policy = new AuthorizationPolicyBuilder().RequireAuthenticatedUser().Build();
        config.Filters.Add(new AuthorizeFilter(policy));
    }).SetCompatibilityVersion(CompatibilityVersion.Version_2_1);

    services.AddHttpContextAccessor();
}

This is my Configure:
    app.UseDeveloperExceptionPage();
    app.UseHttpsRedirection();
    app.UseStaticFiles();
    app.UseCookiePolicy();

    app.UseAuthentication();

    app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });

And I login using this:
[HttpPost]
[Route("/Login")]
[AllowAnonymous]
public async Task<ActionResult> Login(string returnUrl, LoginFormModel loginFormModel)
{
    returnUrl = returnUrl ?? Url.Content("~/");

    if (ModelState.IsValid) {
        var result = await _signInManager.PasswordSignInAsync(loginFormModel.Username, loginFormModel.Password, loginFormModel.RememberMe, lockoutOnFailure: true);
        if (result.Succeeded) {
            _logger.LogInformation("User logged in.");
            return LocalRedirect(returnUrl);
        }
   }

}
Expected behavior
With the SlidingExpiration set to true and the ExpireTimeSpan set to 1 hour I would expect that well after 30 minutes I would be able to continue working on the app.
Additional context
Visual Studio 2017
.NET Core SDK (reflecting any global.json):
Version:   2.1.507
Commit:    e8520940d7
Runtime Environment:
OS Name:     Windows
OS Version:  10.0.17763
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\2.1.507\
Host (useful for support):
Version: 2.2.1
Commit:  878dd11e62

I think the reason is that you didn't configure data protection. I will copy part of the text from the official document. You can read the full document here

... If data protection isn't configured, the keys are held in memory and discarded when the app restarts.
If the key ring is stored in memory when the app restarts:

All cookie-based authentication tokens are invalidated.
Users are required to sign in again on their next request.
Any data protected with the key ring can no longer be decrypted. This may include CSRF tokens and ASP.NET Core MVC TempData cookies.

To configure data protection under IIS to persist the key ring, use one of the following approaches:

Create Data Protection Registry Keys
...
Configure the IIS Application Pool to load the user profile
...
Use the file system as a key ring store
...
Set a machine-wide policy for data protection
...

I have tried the Configure the IIS Application Pool to load the user profile option in my own project and it works.

This setting is in the Process Model section under the Advanced Settings for the app pool. Set Load User Profile to True.

The default Idle Time-out (minutes) of the IIS application pool is 20. So if you don't change its value and the website is idle for 20 minutes, the IIS worker process will be terminated. And if you don't configure the data protection, the keys held in memory will be discarded. That explains why the user is redirected to the login page after 20 minutes. For the This is even if the user is continuously active part, I don't think it will happen. Can you do some investigation after configuring the data protection?

@ZekeLu unfortunately that doesn't seem to be the issue.
Published the application on my local IIS and I even configured the Application Pool Idle Time-out to 60 minutes as well as made sure the Load User Profile was set to true but it still asks for a login at least 30 minutes since the last login.
I will continue to debug this but any tips would be appreciated. Even tips on how to debug it.

Do you think it makes a difference that even the application's homepage (basically http://myapp/) requires a login?

When set to True, keys are stored in the user profile directory and protected using DPAPI with a key specific to the user account. Keys are persisted to the %LOCALAPPDATA%/ASP.NET/DataProtection-Keys folder.
The app pool's setProfileEnvironment attribute must also be enabled. The default value of setProfileEnvironment is true. In some scenarios (for example, Windows OS), setProfileEnvironment is set to false. If keys aren't stored in the user profile directory as expected:

Navigate to the %windir%/system32/inetsrv/config folder.
Open the applicationHost.config file.
Locate the <system.applicationHost> element.
Confirm that the setProfileEnvironment attribute isn't present, which defaults the value to true, or explicitly set the attribute's value to true.

Have you check this?
Please follow these steps to determine whether it is a data protection problem:

login your web application
stop the IIS application pool
make sure the dotnet.exe process spawn by the IIS application pool is terminated (the tool Process Explorer is recommended)
start the IIS application pool
access the protected pages of your web application again.

If login is required, it means that something doesn't survive the restart of the web application. It's very likely that it's a data protection problem.
If login is not required, we are sure that it's not a data protection problem.
In the second case, the next step is to check whether the authentication cookies is sent (that's another reason I can think of). You can use some tools to capture the HTTP traffic (Fiddler is recommended). When re-login is required, check the request and see whether the authentication cookies are sent with the request.
If the authentication cookies are not sent, then we know there is something wrong with the cookies.
If the authentication cookies are sent, can you upload your project here so that I can debug it? Please remove all unnecessary source codes and just keep what is required to reproduce this problem.

Following the data protection steps did not require another login.. so it does not seem like a data protection problem. The cookies are also being sent normally so that doesn't seem to be the issue either. I'm removing all the unnecessary code right now and will upload after another test - maybe something will change after the code change. Really appreciate your time with this. Thanks.

Ok so I wanted to report back that I finally found what the issue was.
Seems that I have bumped into this bug.
In short, it's a bug in ASP.NET Core 2.1 if you are implementing your own IUserStore but not implementing IUserSecurityStampStore, the user is logged out after 30 minutes, even though the cookies did not expire.
A quick fix is increasing the SecurityStampValidation interval to a safely long value like so:
services.Configure<SecurityStampValidatorOptions>(o => o.ValidationInterval = TimeSpan.FromHours(10));
I initially approximated the session to 20 - 30 minutes and used "20 minutes" as a keyword in most of the searching. After timing it more precisely today, it is in fact 30 minutes which helped direct me to more relevant search results! Thanks again @ZekeLu for your time.


[core] Fix 4bit serialization issues in case keys not present in state dict (e.g. when loading adapter weights through PEFT)

What does this PR do?
Fixes: https://github.com/huggingface/peft/issues/1095 and other issues related with QLoRA and adapter loading / saving utilities
This PR fixes two things that were supported in previous bnb versions:
1- Loading a QLoRA adapter weight with bias terms in the Linear layer
2- Loading adapter weights on a quantized base model.
model.load_state_dict will recursively call _load_from_state_dict method on each submodule of the model. That method for 4bit layers has been very recently introduced in this commit: https://github.com/TimDettmers/bitsandbytes/commit/76b40a5c9ae708db98e8b4a13249b2806601a387

bias_data = state_dict.pop(prefix + "bias", None)

Might pop None therefore for linear layers with bias terms and when loading adapter weights.
The line self.bias.data = bias_data.to(self.bias.data.device) will fail - this can be reproduced with the snippet below that load an OPT model which contains bias terms on Linear layers.

Relatively similar issue there since from_state_dict calls pop() it can return None in case there is not the corresponding key in the state dict, again this is applicable only when loading adapter weights. The fix is also straightfoward, simply protect the call to from_state_dict by checking if  prefix.rstrip('.') is inside set(state_dict.keys()).
Reproduction
To reproduce, simply run:
import tempfile
import torch
from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained("peft-internal-testing/tiny-OPTForCausalLM-lora", load_in_4bit=True)

with tempfile.TemporaryDirectory() as tmp_dir:
    model.save_pretrained(tmp_dir)
    model = AutoPeftModelForCausalLM.from_pretrained(tmp_dir, load_in_4bit=True)

    dummy_input = torch.LongTensor([[0, 1, 0]])
    out = model.generate(input_ids=dummy_input)

cc @poedator @TimDettmers @Titus-von-Koeller

I will test it today in the context of the 0.41.2 changes.

@younesbelkada , @TimDettmers
I made a competing PR https://github.com/TimDettmers/bitsandbytes/pull/864 - it removes override of _load_from_state_dict() in Linear4bit altogether.
I suggest to consider mine for it solves a few more issues, and also closer to what I used when testing integration with transformers a while ago.

@poedator @younesbelkada
Can this PR now be closed due to the alternate PR 864 being already merged?

Ok! Yes, I'll look into https://github.com/TimDettmers/bitsandbytes/pull/868 this week.


The testapp served on protractortest.org isn't working for ng2

Probably due to the change to load angular2 libs from node_modules directly.

Closed with 094bcb43d59366887c8565db45c5ea162e2414f0.


Minor changes to support modern Objective-C syntax.

NSDictionary  +dictionaryWithObject:obj forKey:key -> @{key : obj}
NSDictionary -objectForKey:key -> dict[key]
NSArray -objectForKey:0 -> [array firstObject] (in AFURLRequestSerialization.m where it makes sense).
4.NSArray +arrayWithObject[x] -> @[x]

We should actually be looking into moving everything to modern syntax... Perhaps this PR could take that on?

I don't think a third party can help with a pull request, can we? It might be better to land this (assuming it's good, I haven't looked) and ask in another issue. This is something I'd be happy to do if someone doesn't get to it first.

@tewha You can add his remote to your local git instance, and branch off his commit and continue.

I think I got all the object subscripting changes in this PR.

@mattshedlick cool. I havent given it a thorough review, but will do so now knowing you think you got them all. I'll try and do an audit as well.

I ran the modern objective-c conversion tool, and ended up with this commit:
https://github.com/AFNetworking/AFNetworking/commit/ff029cf913a0efc08f9fba93574ecf05bea600b2
Any thoughts on that?

I noticed an error from the upgrade two and commented on it; it changed an action to a read only property.

I've made those small updates called out on that commit.
It looks like the conversion tool added NS_DESIGNATED_INITIALIZER in the wrong places as well, causing a few compile warnings. I'll clean those up.

Alright, 67fb333 now has been squashed down to contain all the changes on top of @mattshedlick's commit.

All tests pass locally.
Thanks for the pr!
:beers:


[ts-styled-plugin] } expected

Making a selector for multiple other components currently shows an syntax error (while it should be correct)

Minimal code to reproduce:
const Child1 = styled.div``;
const Child2 = styled.div``;
const Container = styled.div`
    ${Child1}, ${Child2} {}
`;

My Environment:
Editor: VSCode 1.24.0
Extension: vscode-styled-components 0.0.18
OS: Windows 10

Here's the code that controls this: https://github.com/Microsoft/typescript-styled-plugin/blob/f82699d1a0027cb850118adfcdd8cf88203573dc/src/index.ts#L20
And steps to contribute: https://github.com/Microsoft/typescript-styled-plugin#contributing


Need support for registering and disposing IDisposable types

Webapi had RegisterForDispose() as a HttpRequestMessage extension. We need something similar to that.

why not use scoped service in DI?

Related: https://github.com/aspnet/HttpAbstractions/issues/160

why not use scoped service in DI?

Where we have a need for this it's user-provided data. Ex: WebApiCompatShim with HttpResponseMessage. Defining services for this kind of thing doesn't scale is harder than what users are used to.

We were going to do this as well /cc @GrabYourPitchforks @loudej

I think this was handled by #160.


Build/publish fails when project has a & (ampersand) in it during AOT phase

Android framework version
net9.0-android
Affected platform version
9.0.100-preview.6.24328.19
Description
A net9.0-android (but probably also net8.0-android) project with a & (ampersand) character in it can be created and built, except when AOT is involved, which then fails.
Steps to Reproduce
Run these:

dotnet new android -o "android&stuff"
cd "android&stuff"
dotnet build "android&stuff.csproj" -c Release

Note that running ``dotnet build "android&stuff.csproj"` works just fine.
The result is several errors like this:
 android&stuff failed with 1 error(s) (0.8s)
    C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.99.0-preview.6.340\targets\Microsoft.Android.Sdk.Aot.targets(109,5): error :
      Precompiling failed for E:\repos\BugRepros\andr\android&stuff\obj\Release\net9.0-android\android-arm64\linked\android&stuff.dll with exit code 1.
      missing input files on command line
      `aarch64-linux-android-as` takes a subset of arguments accepted by the GNU Assembler (gas) program.
      Accepted options are limited to the ones used by Xamarin.Android and Mono/dotnet AOT, more can be added as-needed.
      Some options are accepted but ignored, either because they are ignored by GAS as well or because they are used by
      Xamarin.Android or Mono/dotnet AOT, but not needed by `llvm-mc`.  All the unsupported options will cause the wrapper
      to fail with an error message.
      Since `llvm-mc` does not support compiling multiple source files at the same time, this GAS behavior is emulated
      by running `llvm-mc` once per each input file and using the `ld` linker in the end to merge all the discrete output
      files into the single file indicated by the `-o` option.
      Command line options are compatibile with GAS version 2.38
      Currently supported options are:
      All targets
         -o FILE            path to the output object file
        --warn              don't suppress warning messages
         -g | --gen-debug   generate debug information in the output object file
      x86/x86_64 targets
        --32                output a 32-bit object [ignored, `llvm-mc` is always invoked for the right target]
        --64                output a 64-bit object [ignored, as above]
      armeabi/arm32 targets
        -mfpu=FPU           select floating-point architecture for the target
      Ignored by GAS and this wrapper
        --divide
         -k
        --nocpp
         -Qn
         -Qy
         -s
         -w
         -X
      Wrapper options, not passed to `llvm-mc`
         -h | --help        show this help screen
         -V                 show version
        --version           show version and exit
      The system cannot find the path specified.
      The system cannot find the path specified.
      AOT of image E:\repos\BugRepros\andr\android&stuff\obj\Release\net9.0-android\android-arm64\linked\android&stuff.dll failed.
      Mono Ahead of Time compiler - compiling assembly E:\repos\BugRepros\andr\android&stuff\obj\Release\net9.0-android\android-arm64\linked\android&stuff.dll
      AOTID 885475ED-118C-B68C-C29B-8D3F5B8FCFAE
      Compiled: 3/3

You can run step (3) with the -bl parameter to get a binlog with more details.
But it seems that the & char is causing some paths to be troublesome with the AOT pipeline. The same project without the & in it works fine too.
Did you find any workaround?
Don't use & char in project name. But there could be other problematic chars as well. Then again, I recently found that the whole Android stack is at best friendly to ASCII chars in paths, but not all ASCII chars (forget Unicode entirely).
Relevant log output
No response

@Eilon so using any character other that [A-Za-z0-9_.] on android is just flat out not going to work. AndroidStudio is very strict about what you can call projects and package names.
I'm not 100% which part of the AOT pipeline this issue is happening, it might be dotnet/runtime. @jonathanpeppers any thoughts.

@dellis1972 indeed, I've experienced many such issues before, including https://github.com/dotnet/android/issues/6925, and I even filed https://issuetracker.google.com/issues/347706983.
I wonder what, in general, could/should we do about such issues? Of course a lot of this is entirely out of our hands because this part of the stack is outside of .NET. If we assume that these issues won't be fixed in any near future, should we do anything to warn/block such cases? For many cases we can massage data to be valid, but for at least some of the Android issues I've seen there's no massaging we can do in the template because the issue happens if any part of the project path has certain characters. We have full control of names of things within the project (such as generated filenames, namespaces, etc.).
Is it worth pursuing this type of problem more?


Component is loaded without scroll

Why this code, the component foo is loaded without scroll?
<template>

    <HelloWorld msg="Welcome to Your Vue.js App"/>

    <LazyHydrate when-visible :trigger-hydration="documentLoaded">
      <foo />
    </LazyHydrate>

</template>

Console:

Hello! Thank you for submitting a bug report. Please add a reproduction link on CodeSandbox or something similar. Thx!
Generally speaking, unfortunately, if you're using a system which uses the Vue template renderer in the background, components are loaded even though they are not visible (but they're still not hydrated / executed). There is a feature request open in the Vue.js repository which might make it possible to change this: https://github.com/vuejs/vue/issues/9847

Closing this because of inactivity and because it's most likely related to https://github.com/vuejs/vue/issues/9847


Token.

Tokenisation experiment successful.
Delhi Metro tokens.
Are Citibank , clarity to Masterkey.
Fall. Of saree.

Bank ATMs at Metro stations are indicative to

Injury to my back by bomb.
Has been reminded to me by this.
It is scratch . Word

Diwali. Comment.
Has led to some press. Panipat.


feat(updater): add YAML support (#33, #748)

Adding support for YAML files. Based on #748. Geared towards Dart/Flutter pubspec.yaml files, but will work with any YAML file using root level version key.

Thanks so much for this!

npm run format:fix will fix the formatting issues - I tried to push a fix commit to your branch, but it seems I don't have permissions.
Anyway, this looks great - thanks for the feature, I'll merge it once it passes the format check

Fixed formatting issues.

I suspect the failure might be due to line ending issues. I’m travelling this week so can’t take a detailed look at the moment, sorry

updated to preserve line endings

Could this yaml version updater perhaps also support updating versions that are located deeper in the yaml document?
If that would be the case, then this PR could supersede the PR that I opened earlier. Which needs to bump the version located under property: info.version.
I don't want to scope creep your work, so maybe it's better to work this detail out under a separate issue later (after this is PR has been merged).
What do you think?

@ixuz I saw your PR already and I wanted to try to integrate a way to do that, but I did not see an easy way to add additional options to the command (to provide a specific path to update). Maybe there is and I missed it.

Ah, apologies - I missed the notification about that PR for some reason. I've been travelling for the past few weeks, back on Friday. I'll take a quick look now.
The options system is a bit annoying in that it isn't clearly separated from the config - so not all options and ways of setting them are equal. This also makes it harder to add new ways of config, as there isn't a super consistent pattern.
There have been a couple of efforts to address this (the last one stalled I believe, possibly because introducing typescript at the same time made it a larger effort).

so maybe it's better to work this detail out under a separate issue later (after this is PR has been merged).

I think this makes sense -  my gut feel is that it would be nice to have a general yaml support, and then perhaps a few specific ones (eg openapi) that could be just specific configurations of the yaml plugin, maybe? I don't know.

Thanks both for your excellent work, by the way! It's a really nice maintainer experience to receive PRs that include documentation and tests 🙌 🙌

Released as 12.3.0. Thanks again!


A selection of Ethernet things

This includes:
Symbol for Microchip LAN8720A ethernet transceiver

Datasheet
Yet another slightly different QFN-24 footprint for the above

Symbol and footprint for the Amphenol RJMG16x series magjacks
(This one doesn't turn up in a Farnell search for "magjack"...)

Datasheet.
The footprint has some especially fiddly dimensions in it. An annoying thing here is that the RJMG16xxx9xx1xx variant has a subtly different pinout from all the others, although it seems like an obscure variant. I can't contemplate how to sensibly represent this in the footprint name.
(The RJMG163118101NR is by far the cheapest and best-stocked of the bunch so I'm going to assume that's the one I should probably use.)

Thanks for these! I'll try to check them properly this evening, but in the meantime I think the Travis failure is just because the magjack symbol references the footprint with a lower-case x but it should be an uppercase X: https://travis-ci.org/adamgreig/agg-kicad/builds/235420663#L183

Right, I had another think about this and reshuffled it to look like this:

The placement of the power pins on the LAN8720A may be somewhat controversial, but it keeps all the PHY-MAC pins on one side of the symbol.
I also changed all the nPIN to ~PIN~ and P/N & A/C to +/- for consistency.
Just need to sort the footprint now...

I've made all your suggested changes now, I think.


Context conflict when running multiple projects with nuxt-vitest

I have a monorepo with a nuxt 3 app and multiple nuxt modules. The nuxt 3 app already has nuxt-vitest tests and it works great with the new RootDir option. But when I try to add nuxt-vitest to a module in the monorepo, it fails with this error:
> vitest --ui --coverage

failed to load config from /home/jordy/repositories/my_repo/packages/my_package/vitest.config.ts
Error: Context conflict
 ❯ checkConflict node_modules/unctx/dist/index.mjs:6:13
 ❯ Object.set node_modules/unctx/dist/index.mjs:40:9
 ❯ initNuxt node_modules/nuxt/dist/index.mjs:2689:11
 ❯ Object.ready node_modules/nuxt/dist/index.mjs:2680:18
 ❯ loadNuxt node_modules/nuxt/dist/index.mjs:2989:16
 ❯ loadNuxt node_modules/@nuxt/kit/dist/index.mjs:534:19
 ❯ startNuxtAndGetViteConfig node_modules/nuxt-vitest/dist/config.mjs:8:16
 ❯ getVitestConfigFromNuxt node_modules/nuxt-vitest/dist/config.mjs:48:15

On their own it works, it's only when running the global vitest workspace, which has all the coverage and UI in one place.
Not sure how to fix this.

Could you provide a reproduction? 🙏

Here is a repo: https://github.com/TECH7Fox/nuxt-vitest-monorepo
This is what I get when I run npm test:
><EMAIL_ADDRESS>test
> vitest run

failed to load config from /home/jordy/repositories/nuxt-vitest-monorepo/packages/project2/vitest.config.ts
Error: Context conflict
 ❯ checkConflict node_modules/unctx/dist/index.mjs:6:13
 ❯ Object.set node_modules/unctx/dist/index.mjs:40:9
 ❯ initNuxt node_modules/nuxt/dist/index.mjs:2689:11
 ❯ Object.ready node_modules/nuxt/dist/index.mjs:2680:18
 ❯ loadNuxt node_modules/nuxt/dist/index.mjs:2989:16
 ❯ loadNuxt node_modules/@nuxt/kit/dist/index.mjs:534:19
 ❯ startNuxtAndGetViteConfig node_modules/nuxt-vitest/dist/config.cjs:15:16
 ❯ getVitestConfigFromNuxt node_modules/nuxt-vitest/dist/config.cjs:55:15
 ❯ async /home/jordy/repositories/nuxt-vitest-monorepo/node_modules/nuxt-vitest/dist/config.cjs:128:7
 ❯ loadConfigFromFile node_modules/vite/dist/node/chunks/dep-e8f070e8.js:64390:24

It would be great to get this to work. So you can test both the custom packages and the nuxt app in a vitest workspace.

FYI, I can't access your reproduction repo.
I think the issue is likely that there are multiple versions of nuxt firing up.

Ah was still private. You should be able to see it now.

having the same issue

@danielroe any estimate on when this can be fixed?
@TECH7Fox did you manage to get a workaround?


[BugFix] Fix metatool total kv and bytes not match item sum

Fixes #30501
This PR fixes some output inconsistencies of metatool stats.
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[x] 2.5
[ ] 2.4

[FE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[BE Incremental Coverage Report]
:disappointed: fail : 0 / 35 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/tools/meta_tool.cpp
0
1
00.00%
[307]

:large_blue_circle:
src/storage/rowset/rowset_meta_manager.cpp
0
3
00.00%
[94, 95, 96]

:large_blue_circle:
src/storage/tablet_meta_manager.cpp
0
31
00.00%
[1463, 1464, 1466, 1467, 1468, 1469, 1470, 1471, 1472, 1473, 1474, 1475, 1477, 1478, 1479, 1480, 1481, 1482, 1484, 1485, 1487, 1488, 1489, 1490, 1492, 1493, 1496, 1515, 1537, 1538, 1589]

@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.0


Floating point exception (core dumped) when running tutorials_example_trainer using gpu

I'm trying build latest tensorflow but I got a Floating point exception as follow:

zhipeng@tu567:~/apps/tensorflow$ bazel-bin/tensorflow/cc/tutorials_example_trainer --use_gpu
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
I tensorflow/core/common_runtime/gpu/gpu_init.cc:118] Found device 0 with properties:
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.7845
pciBusID 0000:82:00.0
Total memory: 7.92GiB
Free memory: 7.84GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:138] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:148] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:868] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:82:00.0)
Floating point exception (core dumped)

Enviroment info

Ubuntu 14.04
GPU: GTX GeForce 1070
CUDA 8.0RC
Cudnn v5
bazel: 0.3.1-jdk7

gdb

(gdb) r --use_gpu
Starting program: /home/zhipeng/.cache/bazel/_bazel_zhipeng/97115ed19a1a63c4345ae364363ad69b/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/tutorials_example_trainer --use_gpu
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

[Switching to Thread 0x7fff4a886700 (LWP 96208)]
0x000055555600f22a in tensorflow::functor::CastFunctor<Eigen::GpuDevice, float, int>::operator()(Eigen::GpuDevice const&, Eigen::TensorMap<Eigen::Tensor<float, 1, 1, long>, 16>, Eigen::TensorMap<Eigen::Tensor<int const, 1, 1, long>, 16>) ()

And it can make successful execution without 'use_gpu' parameter, I can't find the problem.

Sounds like integer division by 0 somewhere, there was same error message when doing tf.div(1, 0), but @girving added some checks in the div/mod kernels

@yaroslavvb Do floating point exceptions occur if you cast 1e100 to int32?

Shouldn't....tf.cast(1e100, tf.int32) gives MAXINT on GPU and MININT on CPU

Not sure what the problem is, then.  @czp1019 Can you compile in debug mode and investigate what exactly is causing the exception?

I'll try compile in debug mode these days, but I'm not sure if this issue is caused by the version of Cudnn or Cuda. @girving

I think so and I will check it, thanks! @yaroslavvb

This seems to be a Cuda/Driver Issue. I tracked this down on my system and it fails in /external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorExecutor.h at line 250:
const int max_blocks = device.getNumCudaMultiProcessors() *
device.maxCudaThreadsPerMultiProcessor() / block_size;
In some cases a block_size of 0 is returned from device.maxCudaThreadsPerBlock();

@zheng-xq Do you know what might be happening?

I am getting the same error as well, although it only occurs about of the third of the time. The rest of the time the program exits normally, but shows nan for many of the output values.
gdb - on failure
(gdb) r --use_gpu
Starting program: /home/reale/.cache/bazel/_bazel_reale/b77777b530a2fcea5d78d328048dc62b/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/tutorials_example_trainer --use_gpu
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

[Switching to Thread 0x7fff99fff700 (LWP 26539)]
0x000055555605c320 in tensorflow::functor::CastFunctor<Eigen::GpuDevice, float, int>::operator()(Eigen::GpuDevice const&, Eigen::TensorMap<Eigen::Tensor<float, 1, 1, long>, 16>, Eigen::TensorMap<Eigen::Tensor<int const, 1, 1, long>, 16>) ()

last 20 lines from running outside gdb
000009/000002 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000005/000008 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
000008/000001 lambda = 2.000146 x = [0.894449 -0.447170] y = [1.789007 -0.894449]
000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
000007/000004 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000008/000001 lambda = 2.000073 x = [0.894438 -0.447192] y = [1.788931 -0.894438]
000009/000002 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000003/000004 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000005/000008 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
000009/000002 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
000008/000001 lambda = 2.000037 x = [0.894433 -0.447203] y = [1.788893 -0.894433]
000007/000004 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000009/000002 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000007/000004 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000009/000002 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000007/000004 lambda =      nan x = [     nan      nan] y = [     nan      nan]
000007/000004 lambda =      nan x = [     nan      nan] y = [     nan      nan]

Environment
Ubuntu 16.04
GTX 1080
CUDA 8.0 RC + Patch
cudnn 5.1
bazel 0.3.1
gcc 5.4.0

Same intermittent error for me (though I don't see nans):
Environment
Ubuntu 16.04
2xTitan X (Pascal)
CUDA 7.5
cudnn 5.1
gdb
gdb bazel-bin/tensorflow/cc/tutorials_example_trainer
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bazel-bin/tensorflow/cc/tutorials_example_trainer...(no debugging symbols found)...done.
(gdb) r --use-gpu
Starting program: /home/maxim/.cache/bazel/_bazel_maxim/703ad90c2d94a2fe53038a3a66b484c5/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/tutorials_example_trainer --use-gpu
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.7.5 locally
Unknown flag: --use-gpu
[Inferior 1 (process 7289) exited with code 0377]
(gdb) r --use_gpu
Starting program: /home/maxim/.cache/bazel/_bazel_maxim/703ad90c2d94a2fe53038a3a66b484c5/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/tutorials_example_trainer --use_gpu
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.7.5 locally
[New Thread 0x7fffe632e700 (LWP 7295)]
[New Thread 0x7fffe5b2d700 (LWP 7296)]
[New Thread 0x7fffe532c700 (LWP 7297)]
[New Thread 0x7fffe4b2b700 (LWP 7298)]
[New Thread 0x7fffdffff700 (LWP 7299)]
[New Thread 0x7fffdf7fe700 (LWP 7300)]
[New Thread 0x7fffdeffd700 (LWP 7301)]
[New Thread 0x7fffde7fc700 (LWP 7302)]
[New Thread 0x7fffddffb700 (LWP 7303)]
[New Thread 0x7fffdd7fa700 (LWP 7304)]
[New Thread 0x7fffdcff9700 (LWP 7305)]
[New Thread 0x7fffdc7f8700 (LWP 7306)]
[New Thread 0x7fffdbff7700 (LWP 7307)]
[New Thread 0x7fffd57f6700 (LWP 7315)]
[New Thread 0x7fffd4ff5700 (LWP 7316)]
[New Thread 0x7fffd47f4700 (LWP 7317)]
[New Thread 0x7fffd3ff3700 (LWP 7318)]
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:06:00.0
Total memory: 11.90GiB
Free memory: 11.75GiB
[New Thread 0x7fffd37f2700 (LWP 7319)]
W tensorflow/stream_executor/cuda/cuda_driver.cc:572] creating context when one is currently active; existing: 0x7fffe0046af0
[New Thread 0x7fffd2ff1700 (LWP 7320)]
[New Thread 0x7fffd27f0700 (LWP 7321)]
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 1 with properties:
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:05:00.0
Total memory: 11.90GiB
Free memory: 11.37GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 1
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 1:   Y Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 0000:06:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:1) -> (device: 1, name: TITAN X (Pascal), pci bus id: 0000:05:00.0)
[New Thread 0x7fffd1fef700 (LWP 7322)]
[New Thread 0x7fffd17ee700 (LWP 7323)]
[New Thread 0x7fffd0fed700 (LWP 7324)]
[New Thread 0x7fffc3fff700 (LWP 7325)]
[New Thread 0x7fffc37fe700 (LWP 7326)]
[New Thread 0x7fffc2ffd700 (LWP 7327)]
[New Thread 0x7fffc27fc700 (LWP 7328)]
[New Thread 0x7fffc1ffb700 (LWP 7329)]
[New Thread 0x7fffc17fa700 (LWP 7330)]
[New Thread 0x7fffc0ff9700 (LWP 7331)]
[New Thread 0x7fffa608e700 (LWP 7332)]
[New Thread 0x7fffa588d700 (LWP 7333)]
[New Thread 0x7fffa508c700 (LWP 7334)]
[New Thread 0x7fffa488b700 (LWP 7335)]
[New Thread 0x7fffa408a700 (LWP 7336)]
[New Thread 0x7fffa3889700 (LWP 7337)]
[New Thread 0x7fffa3088700 (LWP 7338)]
[New Thread 0x7fffa2887700 (LWP 7339)]
[New Thread 0x7fffa2086700 (LWP 7340)]
[New Thread 0x7fffa1885700 (LWP 7341)]
[New Thread 0x7fffa1084700 (LWP 7342)]
[New Thread 0x7fffa0883700 (LWP 7343)]
[New Thread 0x7fffa0082700 (LWP 7344)]
[New Thread 0x7fff9f881700 (LWP 7345)]
[New Thread 0x7fff9f080700 (LWP 7346)]
[New Thread 0x7fff9e87f700 (LWP 7347)]
[New Thread 0x7fff9e07e700 (LWP 7348)]

Thread 36 "tutorials_examp" received signal SIGFPE, Arithmetic exception.
[Switching to Thread 0x7fffa408a700 (LWP 7336)]
0x00005555560df621 in tensorflow::functor::CastFunctor<Eigen::GpuDevice, float, int>::operator()(Eigen::GpuDevice const&, Eigen::TensorMap<Eigen::Tensor<float, 1, 1, long>, 16>, Eigen::TensorMap<Eigen::Tensor<int const, 1, 1, long>, 16>)
    ()
(gdb)

Same here:
Environment
Ubuntu 16.04
1xTitan X (Pascal)
CUDA 8.0 RC
cudnn 5.1
gdb
`tjy@thresh:~/Libraries/tensorflow$ gdb bazel-bin/tensorflow/cc/tutorials_example_trainer
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bazel-bin/tensorflow/cc/tutorials_example_trainer...(no debugging symbols found)...done.
(gdb) r --use_gpu
Starting program: /home/tjy/.cache/bazel/_bazel_tjy/3585b37c520dd48091f052dff46832b7/execroot/tensorflow/bazel-out/local_linux-py3-opt/bin/tensorflow/cc/tutorials_example_trainer --use_gpu
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
I tensorflow/stream_executor/dso_loader.cc:105] Couldn't open CUDA library libcublas.so.8.0. LD_LIBRARY_PATH:
I tensorflow/stream_executor/cuda/cuda_blas.cc:2289] Unable to load cuBLAS DSO.
I tensorflow/stream_executor/dso_loader.cc:105] Couldn't open CUDA library libcudnn.so.5. LD_LIBRARY_PATH:
I tensorflow/stream_executor/cuda/cuda_dnn.cc:3304] Unable to load cuDNN DSO
I tensorflow/stream_executor/dso_loader.cc:105] Couldn't open CUDA library libcufft.so.8.0. LD_LIBRARY_PATH:
I tensorflow/stream_executor/cuda/cuda_fft.cc:343] Unable to load cuFFT DSO.
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] Couldn't open CUDA library libcurand.so.8.0. LD_LIBRARY_PATH:
I tensorflow/stream_executor/cuda/cuda_rng.cc:333] Unable to load cuRAND DSO.

[Switching to Thread 0x7fffb5d36700 (LWP 12963)]
0x0000555556158c1e in tensorflow::functor::CastFunctor<Eigen::GpuDevice, float, int>::operator()(Eigen::GpuDevice const&, Eigen::TensorMap<Eigen::Tensor<float, 1, 1, long>, 16>, Eigen::TensorMap<Eigen::Tensor<int const, 1, 1, long>, 16>) ()
(gdb)
`

Same here
And same kind of result when running gdb.

Hello,
Same problem here as well.  GTX 1080, Ubuntu 14.04, CUDA 8.0, CuDNN 5.1, nvidia driver 367.27.  --num_concurrent_sessions=1 does not seem to change anything for me, either.  With --use_gpu, roughly 1 in 10-15 runs finishes without error.  Without --use_gpu, it works as expected.

I tried to use r0.10 instead of the head of the master branch, and it works correctly.

The faulty values for the grid sizes are caused by a device not being initalized (but passed to the TensorExecutor). Can be fixed with the attached patch to the TensorExecutor.
Apply from tensorflow base directory:
patch ./bazel-tensorflow/external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorExecutor.h < patch.txt
Should fix this issue for now.
patch.txt

Adding @benoitsteiner to review this Eigen patch.

@girving I encountered the same problem just like you guys. it came to me that the exceptions do not occur when the system is under heavy load. so either this Eigen package is assuming cuda initialization order without a guarantee or nvidia is changing their protocol.

@fheide @yangsiwei880813 Can you try with the latest version ? The problem you are facing could be due to a race condition in the device initialization code that that is now fixed.

I'm assuming this is fixed. Please reopen if this isn't the case.

I have the same issue while training u-net. I tried reducing batch size but with no improvements.
The core dumped message pops up just before completing one epoch.
61/62 [============================>.] - ETA: 0s - loss: 0.4864 - dice_coef: 0.5136[1]    24623 floating point exception (core dumped)

exactly same problem with me .have you found any solution? @bibinmjose

@008karan This was happening when I divided the input-image/255. Pass input-image as it is. Let me know if it works

In my case I provided an batch of size 0 by mistake.

In my case I provided a batch of size 0 by mistake.


add glob pattern tests

since globs are regular expressions not file paths, if we're considering using this for glob patterns we should see how this works with the following (these are just examples, there are a ton of edge cases like this):
/foo/{,/,bar/baz,qux}/
/foo[a\\\/]/
/\\*foo/
/foo\\+/

cc @doowb @phated

Sure, let's do it.
Btw, just a quick heads up in case you're trying to install this that the npm registry was having problems when I published this. I have a ticket in with npm support to get it resolved.

I agree with @es128 on this. If glob-parent returns the proper result, it doesn't seem necessary to add extra logic here. Maybe we need to add these patterns to that test suite.

Moved to https://github.com/es128/glob-parent/issues/10