audio-whitenoise.rs cause seg fault on Linux

Running audio-whitenoise.rs causes a seg fault:
CorruptComputer@Ubuntu-GNOME:~/Desktop/SDL2test$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
 Running `target/debug/sdl2test`
AudioSpec { freq: 44100, format: F32LSB, channels: 1, silence: 0, samples: 1024, size: 4096 }
Segmentation fault (core dumped)

Running Ubuntu GNOME 17.04, 64-bit.

Can you run that with RUST_BACKTRACE=1please ? I'm on linux 64bit as well and I don't have any segfault.
It might be related to Alsa or Pulseaudio, so if you have info about the number of audio devices you have or anything particular that might help me debug that, that'd be great.

RUST_BACKTRACE doesn't seem to change anything:
CorruptComputer@Ubuntu-GNOME:~/Desktop/SDL2test$ RUST_BACKTRACE=1 cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
 Running `target/debug/sdl2test`
AudioSpec { freq: 44100, format: F32LSB, channels: 1, silence: 0, samples: 1024, size: 4096 } Segmentation fault (core dumped)

I know I have Realtek HD Audio, but I have no clue what drivers I am using. Whatever came with Ubuntu has been working fine so I never bothered to look at it. Here's my sound devices if it helps:
CorruptComputer@Ubuntu-GNOME:~/Desktop/SDL2test$ lsmod | grep snd
snd_usb_audio         184320  3
snd_usbmidi_lib        32768  1 snd_usb_audio
snd_hda_codec_hdmi     49152  1
snd_hda_codec_realtek    90112  1
snd_hda_codec_generic    73728  1 snd_hda_codec_realtek
snd_hda_intel          36864  10
snd_hda_codec         126976  4 snd_hda_intel,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_codec_realtek
snd_hda_core           81920  5 snd_hda_intel,snd_hda_codec,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_codec_realtek
snd_hwdep              16384  2 snd_hda_codec,snd_usb_audio
snd_pcm               102400  7 snd_hda_intel,snd_hda_codec,snd_usb_audio,snd_hda_core,snd_hda_codec_hdmi
snd_seq_midi           16384  0
snd_seq_midi_event     16384  1 snd_seq_midi
snd_rawmidi            32768  2 snd_seq_midi,snd_usbmidi_lib
snd_seq                65536  2 snd_seq_midi_event,snd_seq_midi
snd_seq_device         16384  3 snd_seq,snd_rawmidi,snd_seq_midi
snd_timer              32768  2 snd_seq,snd_pcm
snd                    77824  37 snd_hda_intel,snd_hwdep,snd_seq,snd_hda_codec,snd_usb_audio,snd_timer,snd_rawmidi,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_usbmidi_lib,snd_seq_device,snd_hda_codec_realtek,snd_pcm
soundcore              16384  1 snd

Silly me, of course Rust doesn't print a backtrace in a segfault ... Unfortunately my debugging power is quite limited here, the best way to debug this would be to execute this program with gdb and try to get a stack trace, or do some println in the code to pinpoint the exact location of the segfault ... Unfortunately since I can't reproduce this, this is impossible for me to debug right now.

@Cobrand are you using Ubuntu?  I've got that set up on my machines so I'm happy to see if I can reproduce if that's the missing variable.

I am unable to reproduce, I'm using the latest master branch on Ubuntu 16.04.  My sound information:
$ lsmod | grep snd
snd_usb_audio         184320  3
snd_usbmidi_lib        32768  1 snd_usb_audio
snd_hda_codec_hdmi     49152  1
snd_hda_codec_realtek    90112  1
snd_hda_codec_generic    73728  1 snd_hda_codec_realtek
snd_hda_intel          36864  10
snd_hda_codec         126976  4 snd_hda_intel,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_codec_realtek
snd_hda_core           81920  5 snd_hda_intel,snd_hda_codec,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_codec_realtek
snd_hwdep              16384  2 snd_hda_codec,snd_usb_audio
snd_pcm               102400  7 snd_hda_intel,snd_hda_codec,snd_usb_audio,snd_hda_core,snd_hda_codec_hdmi
snd_seq_midi           16384  0
snd_seq_midi_event     16384  1 snd_seq_midi
snd_rawmidi            32768  2 snd_seq_midi,snd_usbmidi_lib
snd_seq                65536  2 snd_seq_midi_event,snd_seq_midi
snd_seq_device         16384  3 snd_seq,snd_rawmidi,snd_seq_midi
snd_timer              32768  2 snd_seq,snd_pcm
snd                    77824  37 snd_hda_intel,snd_hwdep,snd_seq,snd_hda_codec,snd_usb_audio,snd_timer,snd_rawmidi,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_usbmidi_lib,snd_seq_device,snd_hda_codec_realtek,snd_pcm
soundcore              16384  1 snd

$ rustc --version
rustc 1.16.0 (30cf806ef 2017-03-10)

I'm beginning to become suspicious this might be a bug somewhere upstream and not in rust-sdl2 itself.

@CorruptComputer is probably running a newer version of SDL2 than I am though.
@CorruptComputer SDL: http://packages.ubuntu.com/zesty/libdevel/libsdl2-dev
My SDL: http://packages.ubuntu.com/xenial/libsdl2-dev
So maybe it's not a system specific problem as much as one that only occurs in SDL 2.0.5.

I'm setting up a 17.04 VM now to see if I can reproduce.

I got a vm with 17.04, I still can't reproduce this.  @CorruptComputer What rustc are you using?

So I may have made a mistake earlier with my VM while testing this.  The VM didn't have audio, the program was ending instantly, and my terminal may not have been wide enough to see the segfault message.  And I've already deleted my VM.  Sadly I'm out of time to work on this at the moment but I will certainly try again tonight.

Ok, so running with some println!'s it seems to die sometime after device.resume();:
fn main() {
    let sdl_context = sdl2::init().unwrap();
    let audio_subsystem = sdl_context.audio().unwrap();

    let desired_spec = AudioSpecDesired {
        freq: Some(44100),
        channels: Some(1),  // mono
        samples: None,      // default sample size
    };

    // None: use default device
    let mut device = audio_subsystem.open_playback(None, &desired_spec, |spec| {
        // Show obtained AudioSpec
        println!("{:?}", spec);

        MyCallback { volume: 0.5 }
    }).unwrap();

    // Start playback
    device.resume();
    println!("device.resume()");
    // Play for 1 second
    std::thread::sleep(Duration::from_millis(1000));
    println!("thread::sleep()");
    {
        // Acquire a lock. This lets us read and modify callback data.
        let mut lock = device.lock();
        println!("device.lock()");
        (*lock).volume = 0.25;
        // Lock guard is dropped here
    }

    // Play for another second
    std::thread::sleep(Duration::from_millis(1000));

    // Device is automatically closed when dropped
}

It gets this far:
CorruptComputer@Ubuntu-GNOME:~/Desktop/SDL2test$ cargo run
   Compiling sdl2test v0.1.0 (file:///home/ngupton/Desktop/SDL2test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.32 secs
     Running `target/debug/sdl2test`
AudioSpec { freq: 44100, format: F32LSB, channels: 1, silence: 0, samples: 1024, size: 4096 }
device.resume()
Segmentation fault (core dumped)

Commenting out both std::thread::sleep()'s seems to make it not seg fault, however nothing ends up playing:
CorruptComputer@Ubuntu-GNOME:~/Desktop/SDL2test$ cargo run
   Compiling sdl2test v0.1.0 (file:///home/ngupton/Desktop/SDL2test)
warning: unused import: `std::time::Duration`, #[warn(unused_imports)] on by default
 --> src/main.rs:5:5
  |
5 | use std::time::Duration;
  |     ^^^^^^^^^^^^^^^^^^^

    Finished dev [unoptimized + debuginfo] target(s) in 0.33 secs
     Running `target/debug/sdl2test`
AudioSpec { freq: 44100, format: F32LSB, channels: 1, silence: 0, samples: 1024, size: 4096 }
device.resume()
thread::sleep()
device.lock()

Does audio-squarewave play correctly ?
I suspect it's simply the input that we give in this example that is garbage, because there is absolutely no change between whitenoise and squarewave aside from the data we're giving to the playback.
Makes sense anyway, the audio sent is just rand() * 2.0 - 1.0 where rand is between 0 and 1 randomly. I don't know if you're familiar with how sound is represented but giving a random value for the sound 44100 times per second isn't something your computer is going to appreciate IMO. I might be wrong, but I don't see any other way.

I actually just changed to AntergOS, I will test it there when I get the chance.


Error when loading pretrained model with num_classes!=1000

I am trying to load  a pretrained model with a different number of classes to apply transfer learning on cifar10. When I run

model = EfficientNet.from_pretrained('efficientnet-b1', num_classes=23)

I got this error:

AttributeError                            Traceback (most recent call last)
 in
----> 1 model = EfficientNet.from_pretrained('efficientnet-b1', num_classes=23)
~/anaconda3/envs/compression/lib/python3.6/site-packages/efficientnet_pytorch/model.py in from_pretrained(cls, model_name, num_classes)
194     def from_pretrained(cls, model_name, num_classes=1000):
195         model = EfficientNet.from_name(model_name, override_params={'num_classes': num_classes})
--> 196         load_pretrained_weights(model, model_name, load_fc=(num_classes == 1000))
197         return model
198
~/anaconda3/envs/compression/lib/python3.6/site-packages/efficientnet_pytorch/utils.py in load_pretrained_weights(model, model_name, load_fc)
294         state_dict.pop('_fc.bias')
295         res = model.load_state_dict(state_dict, strict=False)
--> 296         assert str(res.missing_keys) == str(['_fc.weight', '_fc.bias']), 'issue loading pretrained weights'
297     print('Loaded pretrained weights for {}'.format(model_name))
AttributeError: 'NoneType' object has no attribute 'missing_keys'

Pytorch version 1.0.1.
Can anyone help me?

Hi @CosimoRulli ,
Perhaps it's a PyTorch version thing?
I made a Colab to test and it is working (at least with PyTorch 1.1): https://colab.research.google.com/drive/1WOWwWK73y_fdVOVU_9GZVICB6jTcTnC3

Upgrading PyTorch version solved my problem! Thank you very much!
Is there any sample or hyper-parameters configuration available?

I have the same problem when using EfficientNet b5.

It's a problem with python's version - I am running torch 1.1.0 everywhere, however in python 3.5  I get:
>>> model = EfficientNet.from_pretrained('efficientnet-b7', num_classes=8) Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "/home/vakko/EfficientNet-PyTorch/efficientnet_pytorch/model.py", line 196, in from_pretrained     load_pretrained_weights(model, model_name, load_fc=(num_classes == 1000))   File "/home/vakko/EfficientNet-PyTorch/efficientnet_pytorch/utils.py", line 298, in load_pretrained_weights     assert str(res.missing_keys) == str(['_fc.weight', '_fc.bias']), 'issue loading pretrained weights' AssertionError: issue loading pretrained weights
while in python 3.6 it loads just fine.... any ideas on this?

Same for me. Python 3.5 torch 1.1 - every new loading pretrained weights I have to reinstall torch or efnet.

U can try this:
    from efficientnet_pytorch import EfficientNet     model = EfficientNet.from_pretrained('efficientnet-b5')     model._fc.out_features = 45     print(model)

AttributeError: 'EfficientNet' object has no attribute 'conv1'
I am getting this error, why?


Support os_log

I'm brand new to this project, so I apologize if I've missed something, but it appears that anything logged using XCLogger does not show up in Console for iOS 10+. It looks like they've deprecated NSLog in favor of the new os_log approach. Could XCGLogger be made to detect the iOS version at runtime and issue the log via os_log so it would still appear in Console, perhaps using an approach like the one described here:
http://blog.safedk.com/technology/nslog-os_log-multiple-ios-versions/

The code coud be something like that

import os.log

open class OSLogDestination: BaseQueuedDestination {

    private var _log: AnyObject?
    @available(iOS 10.0, macOS 10.12, tvOS 10.0, watchOS 3.0, *)
    public var log: OSLog {
        get {
            return _log as? OSLog ?? OSLog.default
        }
        set {
            _log = newValue
        }
    }

    open override func output(logDetails: LogDetails, message: String) {
        var logDetails = logDetails
        var message = message

        // Apply filters, if any indicate we should drop the message, we abort before doing the actual logging
        if self.shouldExclude(logDetails: &logDetails, message: &message) {
            return
        }

        self.applyFormatters(logDetails: &logDetails, message: &message)

        if #available(iOS 10.0, macOS 10.12, tvOS 10.0, watchOS 3.0, *) {
            os_log("%@", log: log, type: logDetails.level.osLogType, message)
        } else {
            NSLog("%@", message)
        }
    }
}
@available(iOS 10.0, macOS 10.12, tvOS 10.0, watchOS 3.0, *)
extension XCGLogger.Level {
    var osLogType: OSLogType {
        switch self {
        case .info, .warning: /* warning as info, could be also error*/
            return .info
        case .error, .severe:
            return .error
        case .debug, .verbose:
            return .debug
        case .none:
            return .default
        }
    }
}

@phimage Could you make a PR for that? Maybe that can speed up adoption of this feature.

@DaveWoodCom maybe you can give access to @phimage to help out. I can also help with reviewing PRs as I'm using this library in one of my apps in production.

@phimage Hi Eric.
I don't see any advantage of setting up a GitHub organization. I'd be interested to hear what the benefits would be.
I usually commit useful PRs and fix issues that are pressing in a timely manner, but I try not to push updates more often than are needed. I find it annoying when a library is constantly changing and so I try and keep this one pretty stable and only push out updates sporadically.
I've been saving the os_log feature as a bonus for patreons https://www.patreon.com/DaveWoodX but so far that hasn't earned a single dollar from XCGLogger users. Right now I have to focus on revenue generating projects (I like to eat). Once I ship the upcoming Launch Center Pro, and All the Rings updates, I'll have some time to add additional features here.

@DaveWoodCom I think @phimage mentioned to get access to review and merge PR's to help out with workload what you described in previous comment. I can offer my help as well.

@fassko I'd be open to granting access, but I think I'd want to audit code/PRs first. I think it would be irresponsible to blindly grant access to people I don't know and trust. That's how open source libraries are compromised. I know XCGLogger is used in several major applications, including banking apps, so I'm extra careful about every change made to the library.

Hi @DaveWoodCom ,
I email you about the benefit of organisation a long time ago and how to manage the access. (but maybe I use a wrong email address ;( )

For organisation the main point is to maintain access to third party log destination (FireBase, Logentries, loggly, etc..., or nslog), in a same place. (also extract ObjcExceptionBridging in an other project)
Then it add some credibility to a project. (remove some fears to adopt it)
People want to participate more often.
I am not for giving access to everybody that make a PR, as done by Moya following https://github.com/Moya/contributors , but with all configuration available and github it seems to work pretty well
Grant access to a project do not give access to the code necessarily.
You can allow to only manage issues (close it, ...) for the beginning
You can forbid to commit in main branch so review all the code if you need it.
You can allow PR only on develop branch, merge only in it, etc... So nothing will be in the main branch without your approbation
(Even if you allow commit in main branch, there is no tagged release but if people use "head" code...)
Finally you can see the github profile of people which want access.

@phimage I'd rather overwrite func write(message: String) instead of func output(logDetails: LogDetails, message: String) as otherwise, the logQueue isn't used at all, which is inherent for BaseQueueDestination.


AP-966 Allow admin users to log in via google

What
Jira AP-966
Adds a link on the Admin User log in screen, which allow a person to log in via their Google account rather than by using user name and password.
There was no design for this - so I've just added a very simple link to the page.
Note, that for log in to work, there needs to be an AdminUser with the same email address as the google user. To add admin users go to /support/admin_user.
Most of the work involved refactoring the True Layer login so that it would work alongside the Google login - Device omniauth is a little complicated if two models both use OAuth2. See the Devise Wiki entry "OmniAuth with multiple models"

I've added the environment variables to the helm deploy code - but please note - at the moment I've used google client id and secret generated from my own google developer console. These will need to be changed for MoJ versions. See the omniauth-google-oauth2 gem's README for info on getting these.


sound not working while call kit is active (iOS 17)

I want to play a ringtone when I call but callkit seems to stop the audio session when the callkit is active, I can't play any audio when callkit is active.
Does anyone have any solution to that? I've tried setting the ringtone but that doesn't work either.
That happens only on iOS 17.

iOS 16.7.1 also not working


Localized language code extraction

The extract_langcode method only extracts localized language codes for "en-us" and "pt-br". langs.json probably needs to be extended to parse more variants (i.e. en-uk, en-au, fr-ca, es-mx...). Or maybe a separate mapping of country names to codes to parse separately? It may be more generically useful to be able to parse locales separate from languages for things like default units, currencies, better location parsing

From Matrix discussion:
extract_langcode("American English") = 'en'
extract_countrycode("American English") = "us"

"American English" or "Inglês Americano" both resolve to US, skills for sure wont have a list for every country in every language

Considerations:

country codes can be ISO Alpha-2, Alpha-3, and Numeric Country Codes.
The new methods should have a flag to support all of these
Should be json based like the lang utils for ease of translation and portability
what is valid input? do we need a confidence param?
from a language you can return the "main" country, but its basically a fuzzy match ...
from a city you could also return the country, this starts getting more into geolocation territory
we need to extract lat, lon from arbitrary strings, some work is done in the big datetime rewrite
- location awareness is a context clue for other parsers, for example to determine what date is "summer" or "winter"


Remove actively misleading help texts

Add and delete both assumed you want to add or delete an app. We no
longer have apps.
The main command had many examples of how to install gitops and
manage applications. None of that is applicable any more, so remove
any mention of it - we'll probably want to do a something with
installation instructions, but this clears the way to do so.

@ozamosi still references to apps under gitops get:
# Get all applications under gitops control
gitops get apps

# Get last 10 commits for an application
gitops get commits <app-name>


Fix flyout interaction & re-enable multi-flyout caching

Reverts & fixes: https://github.com/Microsoft/pxt/pull/5337
My multi-flyout caching introduced with #5323 broke dragging of blocks from the flyout after it was closed then opened.
This was due to destructive side-effects in the Flyout.hide() code in blockly, which I've proposed a fix for here: https://github.com/Microsoft/pxt-blockly/pull/212
Try the fix here: https://arcade.makecode.com/app/e07373f9b0f4944f2a118372b04b2ae7550353b3-0f3b278df6#editor
Validation:

[x] Ensure leaving event registrations on hide() doesn't cause further issues

confirmed the only events that are registered are: mousedown, mouseenter, mouseleave, mousedown which don't trigger on display: none

[ ] complete a tutorial on IE 11

Found an issue with zooming in & out. Only the active flyout gets zoomed.

[ ] Fix zoom in & out issue

You need to clear the cache whenever a project gets reload as the fly out might change , e.g. an extension adds new blocks.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Daryl Zuniga<EMAIL_ADDRESS>Sent: Wednesday, March 13, 2019 7:19:47 AM
To: Microsoft/pxt
Cc: Peli de Halleux; Review requested
Subject: Re: [Microsoft/pxt] Fix flyout interaction & re-enable multi-flyout caching (#5340)
@darzu commented on this pull request.

In webapp/src/blocks.tsxhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt%2Fpull%2F5340%23discussion_r265147387&data=02|01|jhalleux%40microsoft.com|b8c3835e2847492f759e08d6a7bef312|72f988bf86f141af91ab2d7cd011db47|1|0|636880835902591585&sdata=HWVzpYUz3PxNkGtMtwI1hghIo4to9BiFpBU8ZKZkyok%3D&reserved=0:

@@ -1287,11 +1288,43 @@ export class Editor extends toolboxeditor.ToolboxEditor {
this.showFlyoutInternal_(this.flyoutXmlList);
}

private showFlyoutInternal_(xmlList: Element[]) {

private flyouts: pxt.Map<Blockly.VerticalFlyout> = {};

So far, never. I can't think of a case where we'd want to. This cache doesn't grow unbounded, there's only one one entry per toolbox category. We refresh the contents of the flyouts in these caches when appropriate. Some flyouts that change a lot like variables we don't even cache.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt%2Fpull%2F5340%23discussion_r265147387&data=02|01|jhalleux%40microsoft.com|b8c3835e2847492f759e08d6a7bef312|72f988bf86f141af91ab2d7cd011db47|1|0|636880835902591585&sdata=HWVzpYUz3PxNkGtMtwI1hghIo4to9BiFpBU8ZKZkyok%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD-4KVKBcEuOz1DTEoVTCwpcNzeRZ1pBks5vWQkDgaJpZM4btPKN&data=02|01|jhalleux%40microsoft.com|b8c3835e2847492f759e08d6a7bef312|72f988bf86f141af91ab2d7cd011db47|1|0|636880835902591585&sdata=7LoSz8b0vosaydKB3WXueozasMGLeGmjchEPFR8xb6w%3D&reserved=0.

I think there is already a place where we blow out caches
Get Outlook for iOShttps://aka.ms/o0ukef

From: Peli de Halleux
Sent: Wednesday, March 13, 2019 7:21:36 AM
To: Microsoft/pxt; Microsoft/pxt
Cc: Review requested
Subject: Re: [Microsoft/pxt] Fix flyout interaction & re-enable multi-flyout caching (#5340)
You need to clear the cache whenever a project gets reload as the fly out might change , e.g. an extension adds new blocks.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Daryl Zuniga<EMAIL_ADDRESS>Sent: Wednesday, March 13, 2019 7:19:47 AM
To: Microsoft/pxt
Cc: Peli de Halleux; Review requested
Subject: Re: [Microsoft/pxt] Fix flyout interaction & re-enable multi-flyout caching (#5340)
@darzu commented on this pull request.

In webapp/src/blocks.tsxhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt%2Fpull%2F5340%23discussion_r265147387&data=02|01|jhalleux%40microsoft.com|b8c3835e2847492f759e08d6a7bef312|72f988bf86f141af91ab2d7cd011db47|1|0|636880835902591585&sdata=HWVzpYUz3PxNkGtMtwI1hghIo4to9BiFpBU8ZKZkyok%3D&reserved=0:

@@ -1287,11 +1288,43 @@ export class Editor extends toolboxeditor.ToolboxEditor {
this.showFlyoutInternal_(this.flyoutXmlList);
}

private showFlyoutInternal_(xmlList: Element[]) {

private flyouts: pxt.Map<Blockly.VerticalFlyout> = {};

So far, never. I can't think of a case where we'd want to. This cache doesn't grow unbounded, there's only one one entry per toolbox category. We refresh the contents of the flyouts in these caches when appropriate. Some flyouts that change a lot like variables we don't even cache.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt%2Fpull%2F5340%23discussion_r265147387&data=02|01|jhalleux%40microsoft.com|b8c3835e2847492f759e08d6a7bef312|72f988bf86f141af91ab2d7cd011db47|1|0|636880835902591585&sdata=HWVzpYUz3PxNkGtMtwI1hghIo4to9BiFpBU8ZKZkyok%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD-4KVKBcEuOz1DTEoVTCwpcNzeRZ1pBks5vWQkDgaJpZM4btPKN&data=02|01|jhalleux%40microsoft.com|b8c3835e2847492f759e08d6a7bef312|72f988bf86f141af91ab2d7cd011db47|1|0|636880835902591585&sdata=7LoSz8b0vosaydKB3WXueozasMGLeGmjchEPFR8xb6w%3D&reserved=0.

Try adding an extension, open fly out, remove extension, open fly out.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Daryl Zuniga<EMAIL_ADDRESS>Sent: Wednesday, March 13, 2019 7:25:26 AM
To: Microsoft/pxt
Cc: Peli de Halleux; Review requested
Subject: Re: [Microsoft/pxt] Fix flyout interaction & re-enable multi-flyout caching (#5340)
@darzu commented on this pull request.

In webapp/src/blocks.tsxhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt%2Fpull%2F5340%23discussion_r265149999&data=02|01|jhalleux%40microsoft.com|cf1e09a32e194516bb4808d6a7bfbc8c|72f988bf86f141af91ab2d7cd011db47|1|0|636880839276740407&sdata=zQugB6omCltx8RdwZ1OtE1Uv2Q7k24P%2Fnvfoey9LvNI%3D&reserved=0:

@@ -1287,11 +1288,43 @@ export class Editor extends toolboxeditor.ToolboxEditor {
this.showFlyoutInternal_(this.flyoutXmlList);
}

private showFlyoutInternal_(xmlList: Element[]) {

private flyouts: pxt.Map<Blockly.VerticalFlyout> = {};

It's not exactly a cache. We're just maintaining multiple flyouts at once, and they're mutable.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt%2Fpull%2F5340%23discussion_r265149999&data=02|01|jhalleux%40microsoft.com|cf1e09a32e194516bb4808d6a7bfbc8c|72f988bf86f141af91ab2d7cd011db47|1|0|636880839276750415&sdata=OuMyVZRNvJj2nUQfhVflX8XUYyUOdfrzr77IGX0oUtM%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD-4KYmzW890raY0JLypf3_xQzSgeAKmks5vWQpWgaJpZM4btPKN&data=02|01|jhalleux%40microsoft.com|cf1e09a32e194516bb4808d6a7bfbc8c|72f988bf86f141af91ab2d7cd011db47|1|0|636880839276760423&sdata=20U8NyCDa%2BS0S5OY73egr%2FpGB4ZR1LdDGzikcAlb1E4%3D&reserved=0.

Try adding an extension, open fly out, remove extension, open fly out. Get Outlook for iOShttps://aka.ms/o0ukef

I see. I forgot extensions could add blocks to existing flyouts

Also after creating a new variable, blocks might change to autoincrment the block name.

Also after creating a new variable, blocks might change to autoincrment the block name.

I can't find an example of this outside the Variables flyout (which works). Do you know of any?

Also after creating a new variable, blocks might change to autoincrment the block name.

I can't find an example of this outside the Variables flyout (which works). Do you know of any?

Ah found it. Creating the variable "mySprite" causes the sprite blocks to increment to "mySprite2". This is subtle stuff.

I've implemented a finer-grain cache invalidation scheme based on the hash of the blocks xml. This seems to work very well and handles all cases I know of.
Test here:
https://arcade.makecode.com/app/a5970054f60bc4ab93403bc282041065a6ad70db-205a1c8059

[ ] pull in the latest pxt-blockly changes

Found an issue with the caching only in IE.

[ ] fix IE issue

I think you should still clean the cached when a header is reloaded.

Why is that? Since this is caching based on the hash of the blocks xml, it should be plenty precise and will save extra work. The first load can be really slow, on the order of several seconds, up to 10 seconds. Avoid re-doing that work as much as possible is very good thing in my mind. I don't see a case where invalidating on header reload would catch anything that invalidating on the xml hash wouldn't.

Found an issue with the caching only in IE. Works in Firefox, Safari, and Chrome..

[x]  fix IE issue

Fixed. Here: https://arcade.makecode.com/app/fbaae575ebff2f139efa7b5b80c0a4c040cb371c-4f44116f61#editor

Test here: https://arcade.makecode.com/app/6b23f6639d69a49656d0af6dc6495a1b82019220-0bbe4f1578


DBT-version

I want to update to an higher DBT version, but your adapter won't let me, can you update your adapter to at least 1.3, but can you make it flexible? So I can choose wich version of DBT I want to use between 0.20 and 1.3 (the current version of dbt-synapse).

Hi @rvanlingen ,
I have created a new fork from this repository. Looks like it has been abandoned for a long time. Mine fork supports DBT 1.3.x and be able to work with dbt-utils and conduct unit testing.
You can find my forked version at https://github.com/daihuynh/dbt-synapse-serverless/


Not an issue but a question

I am trying to implement Outcomes v2 in my application. The consumer I am testing against is moodle. I have done the folllowing:
LtiLibrary.Core.Outcomes.v2.LisResult result = new LtiLibrary.Core.Outcomes.v2.LisResult();
LtiLibrary.Core.Outcomes.v2.LisPerson person = new LtiLibrary.Core.Outcomes.v2.LisPerson();
result.Comment = “Good Job”;
result.ResultAgent = person;
result.ResultScore = “50%”;
result.TotalScore = 5;
Exception ex = LtiLibrary.Core.Outcomes.v2.OutcomesClient.PostResult(result, Request.Form[“lis_outcome_service_url”].ToString(), Request.Form[“oauth_consumer_key”].ToString(), clsLtiParams.SharedSecret).Exception;
My question is where does the parameters defined in lis_result_sourcedid go. I understand that currently there is no reference to a user in the LMS. How do i do this?

Outcomes 2 is still is draft form, so it is very likely to change. But as it stands now, you post the result to the ResultServiceUrl (not the lis_outcome_service_url). And the ResultServiceUrl will have have the equivalent of the lis_result_sourcedid embedded somewhere in the URL. As a tool provider, you don't need to keep track of it. The flow goes something like this:

When you configure your external tool in Moodle, add a custom parameter for Moodle to pass the ResultServiceUrl to you. For example, custom_result_service_url=$Results.url. If Moodle supports Outcomes 2.0, it will replace $Results.url with the ResultsServiceUrl that is specific to this particular launch.
When you are ready to send a result to Moodle, use PostResult(result, Request.Form["custom_result_service_url"].ToString(), ....);
Because Moodle generated a specific ResultsServiceUrl for the launch, it will know where to save the result.

I checked in with IMS and they will release the new Assignment and Grade Services 2.0 spec early next year (2018). I have not seen the draft, but the person I communicated with thought it was very different than Outcomes 2.0.
https://www.imsglobal.org/activity/learning-tools-interoperability
I'm going to close this issue now. We we can continue the discussion next year if needed.


Treasury Claiming - SubDAO Allocation

If a treasury is allocated x amount of parent tokens - then the treasury has the right to claim x amount of child tokens. As it is now - anyone can claim on behalf of the treasury. Therefore - there will be x amount of tokens which get transferred to the treasury.
Here is the problem - how is this displayed on the front-end - how should this be deposited so it shows up on the frontend.

@christopherdancy @tbwebb22 @adamgall I believe this was talked more about recently in discord. My question here is something that needs to be figured out done sooner or later?

I'm not quite sure exactly the issue being described here.

If a treasury is allocated x amount of parent tokens - then the treasury has the right to claim x amount of child tokens.

I don't follow this premise. @christopherdancy are you saying that during a subDAO creation, if a child DAO is being allocated with some amount of the parent DAO's tokens, then the parent DAO has a right to claim some of the child DAO's tokens? Why is that the case?
The case in which a parent DAO has a right to claim some child DAO tokens, IMO, occurs not when the parent DAO is sending it's own tokens to the child DAO, but when the child DAO is allocating some percentage of it's own tokens to existing parent DAO token holders.

@adamgall Correct - During the funding of child DAO, there is no need to worry about claim rights. However, when we speak about a Parent treasury which owns Parent tokens at the time the snapshot is captured has to worry about claim rights.
To clarify - the parent's treasury may still claim the tokens - however, the frontend will not know which tokens have been sent to the treasury unless the deposit event is emitted.

To clarify - the parent's treasury may still claim the tokens - however, the frontend will not know which tokens have been sent to the treasury unless the deposit event is emitted.

Ah yes, this seems to be the crux of what you're getting at, is it not?
@christopherdancy  what does the token flow look like as tokens are minted, allocated to parent token holders, and claimed? What I'm trying to get at is, how can we make it so that those tokens are eventually deposited to the treasury via the treasury's deposit functions.

@adamgall yes ser it is!
The current flow is as follows - SubDAO created w/ pToken Allocation - If the treasury owns pTokens at the time of creation, the treasury will have a claim to x cTokens - Anyone may claim on the behalf of the treasury, however, x Tokens will not be deposited via the deposit method.
A couple solutions are as follows - allow a user to call an arb tx. via the treasury contract or modify the claim contract to call an arb tx. Either way - we will need to think through the possible side affects. I will review this tomorrow and think through other possible solutions.

Yes - the new contracts do not address this in any way

Well as I remember - we are souring token events based on transfer events and this provided by the gnosis api... no need to even worry about this any longer.

cool cool, I'll close this one out then


Files do not restore their preferred resource when reloading window

VSCode Version:  1.48.0-insider and 1.47.1
OS Version: Darwin x64 18.7.0 / macOS 10.14.6

If I change the casing of a filename in vscode on macOS, the old file name casing is forever stored in ~/Library/Application Support/Code/workspaceStorage, this causes problems with extensions that require case-sensitive file paths
Steps to Reproduce:

In vscode, create a file named EXAMPLE.txt
Rename this file to Example.txt
Right click on the filename in the top bar to see the case hasn't changed everywhere in vscode

Extensions that require case-sensitive file paths will no longer work for this file. This persists through restarts and cache clearing. The only way to restore functionality is by deleting the workspace storage file in ~/Library/Application Support/Code/workspaceStorage

This has come up when using the haxe language extension – after changing a file's case, the old file case given by vscode is sent to the haxe compiler, breaking autocomplete for this file (haxe nightly build 4.2)

Does this issue occur when all extensions are disabled?: Technically Yes (but it's probably not an issue without extensions)

@haxiomic isn't it enough to close the file and open it again? Having a hard time to understand how the casing of the file is persisted even when you close the file. What storage key do you still see the old casing persist?

@bpasero closing the file and opening has no affect, I’ve also tried

restarting vscode
deleting the file and recreating it with the same name (with any variation of casing)
clearing editor history
deleting vscode cache files manually

The extension still receives the original incorrect name
The problem is original name is always kept somewhere inside the workspaceStorage folder, so the only workaround I’ve found is to delete this folder manually

@haxiomic without knowing the exact storage key that the extension is trying to access it is hard to understand the issue. It is possible that maybe the extension stores the URI?

Looks to be in codelens/cache2
I ran through the repro steps, renaming EXAMPLE.hx to Example.hx, in state.vscdb, the name is Example.hx everywhere except codelens/cache2:
{
    "file:///Users/geo/Downloads/haxe-name-bug/EXAMPLE.hx": {
        "lineCount": 9,
        "lines": []
    }
}

I don't know a lot about codelens at this point but this is the jsonrpc object pass in the vshaxe extension for autocomplete:
{
    "jsonrpc": "2.0",
    "id": 17,
    "method": "display/completion",
    "params": {
        "file": "/Users/geo/Downloads/haxe-name-bug/EXAMPLE.hx",
        "contents": "class Example {\n\n\tstatic var x:Int;\n\n\tstatic public function main() {\nxx\n\t}\n\n}",
        "offset": 72,
        "wasAutoTriggered": false,
        "meta": [":deprecated"]
    }
}

Thanks for finding out about the key.

This doesn't make sense. The codelens/cache2 cache is to persist info about where code lenses have been so that you don't see flicker after restarts or when changing files. However, is doesn't define the path of the model so it cannot be cause such issues.

Looks like the editor input serialised and restores the wrong uri

I cannot reproduce an issue here. My steps are the following:

open a file super.ts
notice how codelens caches super.ts because it uses the text model URI
rename super.ts to SUPER.ts
since we cache canonical URIs in memory, for as long as VSCode is open, the URI will be super.ts
in addition, since the editor input was created with the canonical URI, we store and restore this URI, the other one is just used as label information

To get out of this you will have to:

close the editor (needed so that the next time you open the editor it will use the new URI)
reload the window (needed because we persist canonical URIs in memory)
open the editor and the URI is now correct and codelens cache is also correct

I am thinking this just works as designed and is a consequence of all the work we did to fix https://github.com/microsoft/vscode/issues/12448
@jrieken

Looks like the editor input serialised and restores the wrong uri

Well, not sure I would agree? You store canonical URIs in memory, so when the editor input gets created, even when the user changed the URI, the input will get your canonical form. I will store and restore only this form because for the editor input that is the only correct URI.

Has nothing to do with code lens. Do the following

have an extension that does this: vscode.workspace.onDidOpenTextDocument(d => console.log(d.uri.toString()))
open sample.txt in an editor, notice how its path is logged to the console
rename the file to sAMple.txt, the editor title updates, keep the editor open
reload the window
:bug: the extension logs the old name

File editor inputs now keep track of their preferredResource. This is the resource that was originally used by clients when wanting to create an input but it might be different from the resource given our canonical form. However, when the file editor input is being recreated from storage, it will use preferredResource and thus has a chance of being created with the "correct" URI.
This still requires a restart of VSCode but you no longer have to close the editor to get the "correct" URI.

Cannot do this, I see data loss when the file is dirty:

open a file
make it dirty
rename the file to different case
reload window
=> data gone

Very tricky, not having a good solution for now. The truth is that backup identifiers are the hash of a file path, so when an editor opens and gets dirty, the backup is a hash of its file path. When the file is renamed, the hash does not change because we no longer change the underlying model.
If we now simply go ahead and change the model URI on restart because we pick the preferred URI now, the backup can no longer be found and the data is lost. Somehow the backup service would also need to be aware of a preferred URI vs initial URI.
The workaround for now is as I stated before:

close the editor
reload window
open the editor

Would it make sense to validate the persisted URI before making it the canonical one? Maybe not the whole path but just the basename?

@jrieken can you clarify what that would mean?

When restoring an editor input from a case-insensitive file system, check for its actual casing before proceeding to open.

Unfortunately there is no straightforward solution to know what casing a file has on disk other than doing readdir for each folder segment of a path and then using the name that is returned to build up the path.
node.js recently added fs.realpath.native but from my knowledge, this e.g. would convert a Windows subst path to its real path from C: drive, which is not really what we want.
And even if we decide to figure out the real path as it is on disk and then proceeding to create a model with that path, backups would still be stored under the previous case.
I think it is not impossible to fix this, e.g. we have a method to iterate over all backups and could then use the proper IURIIdentityService.extUri to compare for a backup instead of relying on the strict equalness. But since I leave on vac soon, I would only do this change when I am back.
Bottom line: I think the backup service should take the OS path case into account when answering the question wether a backup exists for a resource or not.


Bundle / package / distribute as desktop app (with tauri?)

The app is currently installable as a PWA, which offers a decent UX for android, but still kind of sucks for desktops.
I want users to:

host their own atomic-server
see atomic data in their system tray when running
get notifications
Have minimal impact on performance / memory
quickly open the search bar for atomic (this will be very important later on)
have a single 'atomic' installer for everything that is very easy to use

These things require a different way of bundling / distributing a desktop app.
Tauri
tauri (setup link) could be really interesting as a platform for distributing this app, especially if we can combine it with atomic-server in one package.
That means we can have a single .exe or .dmg file that just works, has a tray icon, and more...

Have been using tauri for a day, and have made a lot of progress. Absolutely amazing tool.
However, I now feel kind of stuck. I'm trying to run a Rust server powered by actix from a tauri application. The problem is there are two async runtimes, and I think they might be incompatible. Every time I try to run async code, it simply does not execute.
I first tried calling my async server creator inside a  tauri spawn:
  tauri::async_runtime::spawn(async {
    atomic_server_lib::serve::serve(&config_clone)
      .await
      .expect("could not start server");
  });

But std::rc::Rc<[std::boxed::Box<(dyn actix_web::data::DataFactory + 'static)>]> cannot be sent between threads safely
So I've tried using the actix runtime in tauri main.rs:
#[actix_rt::main]
async fn main() {
  actix_rt::spawn(async move {
    println!("This will not print");
  });

But now whatever is inside spawn will never even run.
Then I have one more trick up my sleeve:
  let system = actix_rt::System::new("atomic-server");
  actix_rt::spawn(async move {
    println!("I'm printing just fine")
  });
  system.run().unwrap();

Which does run, but after calling system.run(), nothing happens, as it's a blocking function that loops forever, of course.  Perhaps if I could call system.run() from within the tauri runtime, things could work...
  tauri::async_runtime::spawn(async move {
    let _r = system.run();
  });

... but I can't call that function in a tauri::async_runtime::spawn block.
Is there a way to combine both async runtimes?

The beta version of actix will use the tokio runtime, which means that this problem should be adressable. So I've tried updating the actix deps, but it won't compile, as both tauri and actix use different linked versions of zstd:
error: failed to select a version for `zstd-sys`.
    ... required by package `zstd-safe v4.1.1+zstd.1.5.0`
    ... which satisfies dependency `zstd-safe = "=4.1.1"` of package `zstd v0.9.0+zstd.1.5.0`
    ... which satisfies dependency `zstd = "=0.9.0"` of package `tauri-codegen v1.0.0-beta.4`
    ... which satisfies dependency `tauri-codegen = "=1.0.0-beta.4"` of package `tauri-macros v1.0.0-beta.5`
    ... which satisfies dependency `tauri-macros = "=1.0.0-beta.5"` of package `tauri v1.0.0-beta.8`
    ... which satisfies dependency `tauri = "=1.0.0-beta.8"` of package `atomic-server-tauri v0.28.3 (/Users/joep/dev/github/joepio/atomic-data-rust/src-tauri)`
versions that meet the requirements `=1.6.1` are: 1.6.1+zstd.1.5.0

the package `zstd-sys` links to the native library `zstd`, but it conflicts with a previous package which links to `zstd` as well:
package `zstd-sys v1.5.0+zstd.1.4.9`
    ... which satisfies dependency `zstd-sys = "=1.5.0"` of package `zstd-safe v3.1.0+zstd.1.4.9`
    ... which satisfies dependency `zstd-safe = "=3.1.0"` of package `zstd v0.7.0+zstd.1.4.9`
    ... which satisfies dependency `zstd = "^0.7"` of package `actix-http v3.0.0-beta.12`
    ... which satisfies dependency `actix-http = "=3.0.0-beta.12"` of package `actix-files v0.6.0-beta.8`
    ... which satisfies dependency `actix-files = "=0.6.0-beta.8"` of package `atomic-server v0.28.2 (/Users/joep/dev/github/joepio/atomic-data-rust/server)`
    ... which satisfies path dependency `atomic-server` of package `atomic-server-tauri v0.28.3 (/Users/joep/dev/github/joepio/atomic-data-rust/src-tauri)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the links ='zstd-sys' value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

failed to select a version for `zstd-sys` which could resolve this conflict

This seems to fix the issue above: https://github.com/actix/actix-web/commit/68a3acb9c290945eab82feeb87a98c47abcf0fac

Updating actix fixed compilation, but unfortunately not my initial problem of runtimes.
I've specified my problem in tauri discussion board: https://github.com/tauri-apps/tauri/discussions/2942

Ok, the basis is set up. Some work is left to do, but I'll open new issues for that.


Libasync (linux) - Make sure TCP write ready events always occur

This is a pretty bad bug actually, the protocol freezes when write blocks. Should be merged urgently.

Tested in production as well

I didn't merge this earlier because I didn't understand why a write operation would depend on a "read ready" event. Do you have any further insights on this? Anyway, this is safe to merge, so I'll include it in the release.

I also have an issue understanding why I was blocking on write and receiving only a read event


Rename: Add support for modification function

closes #266

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (0fca756) 94.68% compared to head (9b47921) 94.62%.

Files
Patch %
Lines

src/transforms/rename.jl
90.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #267      +/-   ##
==========================================
- Coverage   94.68%   94.62%   -0.06%
==========================================
  Files          46       46
  Lines        1317     1322       +5
==========================================
+ Hits         1247     1251       +4
- Misses         70       71       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


fix(biome_css_parser): fixed css @document parser error

Summary

Closed: https://github.com/biomejs/biome/issues/4258
Test Plan

Add test cases:
@-moz-document url-prefix() {}
@-moz-document url-prefix("https://www.example.com/") {}
@-moz-document url-prefix() {
	body {
		background-color: green;
	}
}
@-moz-document url-prefix("https://www.example.com/") {
	body {
		background-color: green;
	}
}

@chansuke I have no idea about the ci error, it seemed that the git check didn't check any change in my pr?


Added support for specifying database Schema name.

See Issue "provide table schema #63

Seems like a good addition.
Any possible version/portability/permission considerations with the switch to sys.tables usage? Not sure what the minimum version of SQL Server currently supported by this sink is.

I think sys.tables has been pretty stable, at least for 2008 and on (not that I think older versions aren't compatible, it's just that I can't even remember what the version was before that).
Might be worth putting a line in the sand and stating a minimum version, but that would require testing across versions. I'd like to see the unit tests updated to cover this explicitly (I think they're going to catch any problems implicitly). Even though we're not able to run them automatically, it would it easier to test on a dev machine.

According to MSDN: https://msdn.microsoft.com/en-us/library/ms187406.aspx

SQL Server (starting with 2008)

As far as permissions: https://msdn.microsoft.com/en-us/library/ms187113.aspx

The visibility of metadata is limited to securables that a user either owns or on which the user has been granted some permission. For example, the following query returns a row if the user has been granted a permission such as SELECT or INSERT on the table myTable.

So anyone with SELECT or INSERT will have access to it. Which should be fine.

it's just that I can't even remember what the version was before that

@colin-young my thoughts as well :-)
SQL Server 2005 is still used out there - I wonder if anyone relying on this package is using it?
Since the change only impacts automatic table creation, and there's a schema script available separately, I'd say the risk of anyone being majorly inconvenienced is pretty low.
Since the added parameter is a breaking change anyway, perhaps the major version bump has this covered.
:shipit:?

@nblumhardt That seems like a perfectly reasonable approach.

:+1: - this will appear as 5.0.0-dev-nnnnn.


Cannot --export via command line in 3.1.1 (mono)

Godot version: 3.1.1.stable.mono.official
OS/device including version: Windows 10
Issue description:
When trying to export a project via the command line, I get an error.
Command:
godot --path . --export win64 build/bin/win64/signfordelivery.exe --no-window

Output:
OpenGL ES 3.0 Renderer: Radeon RX 580 Series
Mono: Logfile is: C:\Users\Wouter\AppData\Roaming/Godot/mono/mono_logs/2019_05_03 13.33.45 (11436).txt
ERROR: instance: Class 'EditorSettings' can only be instantiated by editor.
   At: core/class_db.cpp:523
ERROR: poll: C:\Users/Wouter/AppData/Roaming/Godot/editor_settings-3.tres:3 - Parse Error: Can't create sub resource of type: EditorSettings
   At: scene/resources/resource_format_text.cpp:561
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: core/io/resource_loader.cpp:208
ERROR: Failed loading resource: C:\Users/Wouter/AppData/Roaming/Godot/editor_settings-3.tres
   At: core/io/resource_loader.cpp:285
WARNING: create: Could not open config file.
     At: editor/editor_settings.cpp:871
ERROR: _fs_changed: Project export failed with error code 9.
   At: editor/editor_node.cpp:495
debugger-agent: Unable to listen on 4200

This happens with my game but also with a completely new, empty project in 3.1.1. The export command worked fine in 3.1.
Steps to reproduce:

Create a new (empty) project
Create an export template
Export it with the CLI command above

--project . is not a valid argument. I think you mean --path ..

Oops, you're right. My build script does in fact use --path. I goofed when I wrote it here. Updated.

The same happens to me without using mono.
I am using the version of Godot distributed through Steam, so my paths in the error log are a little bit different. My editor version is 3.1.1.stable.official .

I'm seeing a similar set of issues. I'm using Windows 10, trying to export via CLI on the Steam version of Godot.
My command line:
godot.windows.opt.tools.64.exe ..\project\project.godot --export "HTML5" ".\html5\"
When I run this, I get the following results:
OpenGL ES 3.0 Renderer: GeForce GTX 1080/PCIe/SSE2 ERROR: _fs_changed: Project export failed with error code 9.    At: editor/editor_node.cpp:495
I'm not sure what is causing this issue.
On a side note, if I include --path in my command line, it actually creates a new project.godot, overwriting the old one and completely nuking my project settings - I have to discard my changes in VCS to get it back.
Thanks,
-Devin

Hi,
I spent more time playing around with this.
Regarding Project export failed with error code 9:
It looks like the folder has to exist before you can deploy to it. Try making the folder first. Once the export path existed I was no longer getting that error. I see no reason why Godot couldn't make that folder when running --export -- especially if it's going to give an ambiguous error code instead of any usable feedback.
Regarding --path:
Looks like when you exclude the --path option, it expects the actual project.godot file path. When you use --path in your command line, it instead wants the directory, not the filepath, to your project.
Therefore, it should be used like:
godot.windows.opt.tools.64.exe --path ..\project\ ...
OR
godot.windows.opt.tools.64.exe ..\project\project.godot
I feel this behavior is misleading and destructive... shouldn't they be changed to match? Personally I see no reason why both shouldn't require you point directly at the project.godot file...
Thanks,
-Devin

I've investigated further on this, and I'm getting really frustrated.
It looks like a few things:

The path to the project.godot file/directory is relative to the current working directory

This is the one thing that behaves like I'd expect 👍

The output path for the export template must be specified

Why is this necessary? The build templates already have their output locations specified
export_presets.cfg has a export_path key for every build

This is an absolute path?? This means it breaks on any machine that isn't using the exact same path
Can this be a relative path, then I can ensure that I always know where the builds will go
There's no reason the CLI can't still specify an override, of course

The output path for the export template is relative to the project.godot directory

Why am I suddenly changing context? Am I mistaken here?
This makes it near impossible to create intuitive command line deployment scripts

The target directory must already exist

Since I want to export anyways, Godot should ensure directories exist

At the very least this should be an option on the CLI

Paired with the above point, I cannot simply create this directory via the same path provided to the export template (which shouldn't need the path in the first place unless overridden)

I have to either make sure the directory is created ahead of time (not gonna happen on clean instances) or ensure that I create the dir using a different path than the one I'm passing to the export directory????

This has wasted a lot of my time and is quite frustrating, if not embarrassing, that Godot behaves this way (And does not provide feedback, have documentation. Maybe I'm completely missing the memo here?)
Either way, please try your export with my findings and confirm that it's the case or at least confirm I'm going insane :)
Thanks,
-Devin

I have the same exact issue, except the Godot version used is not mono:
Godot_v3.1.1.stable.official
Windows 10
Used command:
..\godot.exe --export "Linux X11" .

Output:
OpenGL ES 2.0 Renderer: GeForce GTX 1050 Ti/PCIe/SSE2
ERROR: instance: Class 'EditorSettings' can only be instantiated by editor.
   At: core/class_db.cpp:523
ERROR: poll: C:\Users/Rokas/AppData/Roaming/Godot/editor_settings-3.tres:3 - Parse Error: Can't create sub resource of type: EditorSettings
   At: scene/resources/resource_format_text.cpp:561
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: core/io/resource_loader.cpp:208
ERROR: Failed loading resource: C:\Users/Rokas/AppData/Roaming/Godot/editor_settings-3.tres
   At: core/io/resource_loader.cpp:285
WARNING: create: Could not open config file.
     At: editor/editor_settings.cpp:871
ERROR: _fs_changed: Unknown export preset: Linux X11
   At: editor/editor_node.cpp:474

@DevinPentecost I was having the same issue earlier today, and my independent investigation led to the same conclusions as yours--it would have saved me some time if I'd found your answer first!

Possibly related to #34148
Path should be relative to project file location.

Can anyone still reproduce this bug in Godot 3.2.3 or any later release?

While I was creating a script for exporting project for different targets I found an export issue, applied for Mac and Godot 3.2.3. If I'm running command like

godot --path . --export "Linux/X11" "maze_export/lin"
it is finishing with
ERROR: copy: Failed to open ../maze-export/lin
At: core/os/dir_access.cpp:307.
ERROR: fs_changed: Project export failed with error code 12 for preset 'Linux/X11'.
At: editor/editor_node.cpp:753.
And I found solution which is working for Linux, Windows and Html5 - add a target with extension, like
godot --export "Linux/X11" $MAZE_EXPORT/$LIN/maze2d-lin64.run
But that is not working for Android.. I can't export resource pack with using of
godot --export "Android" $MAZE_EXPORT/$ANDROID_/maze2d.pck
instead it gives me the same apk file.

Upd: not and issue for Android - resources are backed into apk.

Closing due to lack of response. Please comment if you can still reproduce this bug on the latest Godot version.
PS: If no minimal reproduction project was included in the original bug report, please upload one as well to ease troubleshooting.


Prepare for MainNet

We want Wallet be ready for MainNet. This is Epic that collects all issues that required in preparation.

Will close this one


First draft of an alternative metadata API

This exploration was inspired by discussion https://github.com/saleor/saleor/discussions/9223
It makes working with metadata easier in languages like JS where unmarshalling a JSON structure happens automatically.
New API (we likely need a better field name but metadata is already taken):
query {
  product(id: ...) {
    meta
  }
}

Response:
{
  "data": {
    "product": {
      "meta": {
        "myKey": "myValue",
        "anotherKey": "anotherValue"
      }
    }
  }
}

I declared a new scalar type so strictly typed clients can provide a custom type override like Map<string, string>.
For now, a write/update API is out of scope.
If we like that, we should also implement a similar interface for all Editor.js content.
Tracked under SALEOR-6043
Impact

[ ] New migrations
[x] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[x] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

Related tracker item: SALEOR-6043

CC @benoj

(for anyone who doesn't remember current metadata structure)
So, instead of representing it below format:
metadata: [{key: "1", value: "4", __typename: "MetadataItem"}]
we will have:
meta: {"1": "4"}
Personally, I like the idea, looks much simpler.

LGTM :shipit:

If we all like it, do you think it's worth merging as is (we still need tests, of course)? Any suggestions for a better field name?

@patrys I think meta is slightly ambiguous if I you want to be pedantic about the naming. However I can't think of decent alternatives.
Here are few ideas:
meta_fields - implies that there are only fields?
meta_values - implies that there are only values?
metaJSON - not sure if it is accurate though

Looks good, I'd say just call it metafields and it seems descriptive enough

@patrys could you please also document metafield
query {
  product(id: ...) {
    metafield(key: "foo")
  }
}

@timuric I would need to implement that first.

@eCzerniak I don't see a Clickup task so I'm assigning you here. You should get how to test it from the PR description, but if needed please ask @patrys for instructions.


Official container on DockerHub, AWS Public ECR

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Describe the Feature
Official docker container on dockerhub, ghcr with alpine or scratch
Expected Behavior
Run something like this for ghcr
$ docker run -it --rm \
  ghcr.io/cloudposse/atmos:v1.10.3 \
  1.10.3

Run the same for dockerhub so same above command except the image would be cloudposse/atmos:v1.10.3
Use Case

Be able to run atmos commands without installing
Possibly use spacelift to run an atmos container

Good point. We should have that. I am down for it if you want to take a stab at it.

https://cloudposse.atlassian.net/browse/DEV-904

This was completed in:

#627


fix(iteration service): Address get iteration error on standalone planner

fix(iteration service): Address get iteration error on standalone planner

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616.int.rdu2c.fabric8.io

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616-fabric8-planner.int.rdu2c.fabric8.io

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616-fabric8-planner.int.rdu2c.fabric8.io

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616-fabric8-planner.int.rdu2c.fabric8.io

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616-fabric8-planner.int.rdu2c.fabric8.io

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616-fabric8-planner.int.rdu2c.fabric8.io

@nimishamukherjee snapshot fabric8-ui is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-1616-fabric8-planner.int.rdu2c.fabric8.io


Application using customtkinter is not compiled properly

Hello, I would like to report an issue that I have with Nuitka 1.2.7 and Python 3.10 on Windows 11 when trying to compile my program, which uses customtkinter. This is the minimal code:
import tkinter
import customtkinter

customtkinter.set_appearance_mode("System")  # Modes: system (default), light, dark
customtkinter.set_default_color_theme("blue")  # Themes: blue (default), dark-blue, green

app = customtkinter.CTk()  # create CTk window like you do with the Tk window
app.geometry("400x240")

def button_function():
    print("button pressed")

# Use CTkButton instead of tkinter Button
button = customtkinter.CTkButton(master=app, text="CTkButton", command=button_function)
button.place(relx=0.5, rely=0.5, anchor=tkinter.CENTER)

app.mainloop()

nuitka nuitest.py --standalone --enable-plugin=tk-inter

Traceback (most recent call last):
  File "C:\Users\trans\PYCHAR~1\NADO-M~1\MICROW~1.DIS\nuitest\NUITES~1.DIS\customtkinter\windows\widgets\theme\__init__.py", line 5, in <module customtkinter.windows.widgets.theme>
  File "C:\Users\trans\PYCHAR~1\NADO-M~1\MICROW~1.DIS\nuitest\NUITES~1.DIS\customtkinter\windows\widgets\theme\theme_manager.py", line 18, in load_theme
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\trans\\PYCHAR~1\\NADO-M~1\\MICROW~1.DIS\\nuitest\\NUITES~1.DIS\\customtkinter\\windows\\widgets\\theme\\../../../assets\\themes\\blue.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\trans\PYCHAR~1\NADO-M~1\MICROW~1.DIS\nuitest\NUITES~1.DIS\nuitest.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "C:\Users\trans\PYCHAR~1\NADO-M~1\MICROW~1.DIS\nuitest\NUITES~1.DIS\customtkinter\__init__.py", line 10, in <module customtkinter>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "C:\Users\trans\PYCHAR~1\NADO-M~1\MICROW~1.DIS\nuitest\NUITES~1.DIS\customtkinter\windows\__init__.py", line 1, in <module customtkinter.windows>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "C:\Users\trans\PYCHAR~1\NADO-M~1\MICROW~1.DIS\nuitest\NUITES~1.DIS\customtkinter\windows\ctk_tk.py", line 9, in <module customtkinter.windows.ctk_tk>

When I manually copy darkdetect and customtkinter, the error changes to the following, probably because customtkinter is ignored for tkinter:
Traceback (most recent call last):
  File "C:\...microwallet.py", line 218, in <module>
AttributeError: 'CTkLabel' object has no attribute 'set_text'

If I remember correctly, I was able to use this in Python 3.8 without much difficulty. Might be before --follow-imports was used instead of --standalone, but my memory may betray me here.
I have also tried to use --include-package-data and --include-data-files for customtkinter and darkdetect (dependency) but to no avail.
My Nuitka version:
1.2.7
Commercial: None
Python: 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Flavor: Unknown
Executable: C:\Program Files\Python310\python.exe
OS: Windows
Arch: x86_64
WindowsRelease: 10

On Windows 11, I have installed Python 3.10 using the official installer from https://www.python.org/downloads

python -m pip freeze

absl-py==1.3.0
aiohttp==3.8.1
aiosignal==1.2.0
ansicolors==1.1.8
astunparse==1.6.3
async-timeout==4.0.2
asyncio==3.4.3
attrs==21.4.0
bleach==5.0.1
Brotli==1.0.9
build==0.9.0
cachetools==5.2.0
certifi==2022.6.15
cffi==1.15.1
charset-normalizer==2.1.0
colorama==0.4.6
coloredlogs==15.0.1
commonmark==0.9.1
cryptography==37.0.4
customtkinter==5.0.3
cycler==0.11.0
darkdetect==0.7.1
docutils==0.19
engineering-notation==0.7.0
flatbuffers==22.12.6
fonttools==4.34.4
frozenlist==1.3.0
gast==0.4.0
google-auth==2.15.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.51.1
h5py==3.7.0
humanfriendly==10.0
idna==3.3
importlib-metadata==5.1.0
inflate64==0.3.1
jaraco.classes==3.2.3
keras==2.11.0
keyring==23.11.0
kiwisolver==1.4.3
libclang==14.0.6
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib==3.5.2
more-itertools==9.0.0
msgpack==1.0.4
multidict==6.0.2
multivolumefile==0.2.3
Nuitka==1.2.7
numpy==1.23.0
oauthlib==3.2.2
opt-einsum==3.3.0
ordered-set==4.1.0
packaging==21.3
pep517==0.13.0
Pillow==9.2.0
pkginfo==1.9.2
protobuf==3.19.6
psutil==5.9.2
py7zr==0.20.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybcj==1.0.1
pycairo==1.20.1
pycparser==2.21
pycryptodomex==3.16.0
Pygments==2.13.0
pyparsing==3.0.9
pyppmd==1.0.0
pyreadline3==3.4.1
PySocks==1.7.1
python-dateutil==2.8.2
pytorch3d==0.0.1
pywin32==305
pywin32-ctypes==0.2.0
pyzstd==0.15.3
readme-renderer==37.3
requests==2.28.1
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
rfc3986==2.0.0
rich==12.6.0
rsa==4.9
shiboken2==<IP_ADDRESS>
six==1.16.0
stem==1.8.1
tensorboard==2.11.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-intel==2.11.0
tensorflow-io-gcs-filesystem==0.28.0
termcolor==2.1.1
texttable==1.6.7
tk-tools==0.16.0
tomli==2.0.1
tor==1.0.0
tornado==6.1
twine==4.0.2
typing_extensions==4.4.0
urllib3==1.26.9
webencodings==0.5.1
Werkzeug==2.2.2
wrapt==1.14.1
yarl==1.7.2
zipp==3.11.0
zstandard==0.19.0

My apologies for mixing up things, as Fr0st3h noted, set_text is not a method supported by CTkLabel and the official example does not include its use. Therefore, the problem is unrelated to what is shown in the minimal reproducible example. I was confused by the method working in Python and not in Nuitka, but it can be replaced with .configure(text="")
.

Well, I care about compatibility of course more than what API should be used, so there is merit in this report for sure.

The CPython flavor is finally well detected on Windows as well, thanks for pointing out what you have there. I indeed agree very much that whatever is working with Python ought to work with Nuitka too. From what I found adding the data files is sufficient.
I have optimized the plugin interactions between Qt and tk-inter a lot too, so this should be very smooth now.

Thanks for your report, this is fixed on the factory branch, which is a development version under rapid development. You can try it out by going here: https://nuitka.net/doc/factory.html
Feedback if this is working is very welcome, just please do not share plans of doing it, but rather confirmations or denials of it working.

This is part of the stable release 1.3 that I just made.


Do away with using URLComponents for urlURL

HTTPServerRequest.urlURL is currently built using URLComponents. Though
convenient, this approach has proven to be convoluted because of percent
encoding restrictions. The alternative is to fall back to building a URL String
, something that we did before adopting URLComponents. We can chose to keep
the percent encoding in HTTPServerRequest.
An additional change is in the initializer of ClientRequest. Though we need
to add percent encoding wherever necessary, we should not percent encode
an percent-encoded path.

Tested this with: Kitura, SwiftMetrics, Kitura-Sample and Kitura-CouchDB

We're basically reverting https://github.com/IBM-Swift/Kitura-NIO/commit/46fbee05dfb9b86d331cb24b1e9e33436a5e1bf3 and a few other commits that fixed regressions of this. Also, the ~2% gain is offset by multiple calls to addingPercentEncoding\removingPercentEncoding


Enable the ability to specify the service target port

Description of the change
Enable the ability to specify the service target port to avoid hardcoded value
Checklist

[x] Chart version bumped in Chart.yaml according to semver.
[x] Variables are documented in the values.yaml and added to the README.md. The helm-docs utility can be used to generate the necessary content. Use helm-docs --dry-run to preview the content.
[ ] JSON Schema generated.
[x] List tests pass for Chart using the Chart Testing tool and the ct lint command.

@sabre1041  lgtm

Will need rebasing with recent merge to master, then we can merge

@ChrisJBurns should be good to go now


Lua 5.3 needs to be bumped to 5.3.6, package Lua 5.4.0

Describe the bug
Lua 5.3.6 was released on 25.09. and we finally received Lua 5.4.0 on 29.06.(!), see https://www.lua.org/news.html and https://www.lua.org/manual/5.4/readme.html#changes.
To Reproduce
Steps to reproduce the behavior:

Check pkgs/development/interpreters/lua-5/default.nix
See that lua5_3 is on release 5, and a lua5_4 is nowhere to be found

Expected behavior
Shiny new version to play with. :stuck_out_tongue_winking_eye:
Screenshots
-
Additional context
-
Notify maintainers
There are no meta.maintainers specified for the Lua interpreters… I'll blindly ping some ppl that look to be relevant to the Nixpkgs Lua ecosystem. (sorry!)
cc @teto @vcunat
Metadata

system: "x86_64-linux"
host os: Linux 4.15.0-112-generic, Ubuntu, 18.04.5 LTS (Bionic Beaver)
multi-user?: no
sandbox: yes
version: nix-env (Nix) 2.3.5
channels(bt1cn): "nixpkgs-20.09pre242495.3c0e3697520"
nixpkgs: /home/bt1cn/.nix-defexpr/channels/nixpkgs

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: [ lua5_3 ]

should be easy enough to add them, duplicate the file and look what's up :) I dont have a specific interest in providing this  but can review if needed.

I think this can be closed.


scm_service_connection gives error when explicit provider host is not set

Describe the bug
When attempting to use the scm_service_connection resource or data object, I get an object not found error.
When looking at the logs I noticed when host isn't set the api call is made to Host: api.strata.paloaltonetworks.com which I would expect it to actually be api.sase.paloaltonetworks.com. When I set the host on the provider it is able to make the api call correctly.
Expected behavior
Create or query a service connection resource correctly without error.
Current behavior
Encountering an error
Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Error reading config
│
│   with data.scm_service_connection.example,
│   on main.tf line 15, in data "scm_service_connection" "example":
│   15: data "scm_service_connection" "example" {
│
│ object not found

Steps to reproduce
TF Repro File
terraform {
  required_providers {
    scm = {
      source  = "paloaltonetworks/scm"
      version = "0.9.2"
    }
  }
}

provider "scm" {
  logging = "detailed"
#   host = "api.sase.paloaltonetworks.com"
}

data "scm_service_connection" "example" {
   id = "REDACTED"
   folder= "Service Connections"
}

Create a service connection manually (or alternatively attempt to create one) without host set on the provider
Run terraform plan
Witness error
Set host to be "api.sase.paloaltonetworks.com"
Witness successful run

Screenshots

Context
I am also witnessing this behavior with remote_networks. I suspect this might be coming from this line
Your Environment

Version used: 0.9.2
Environment name and version (e.g. Chrome 59, node.js 5.4, python 3.7.3):
Operating System and version (desktop or mobile):
Link to your project:

The documentation for the default value for the "host" param within the provider block was incorrect.  The default value for "host" is "api.strata.paloaltonetworks.com", so the documentation for this can be updated to reflect the correct default value.

closing as this is by design. Thanks!


IntVar == None Segfaults

I don't think this is the intended behavior. Comparison between an IntVar and None results in a SegFault. What is extra strange is that using 'is' instead of '==' doesn't segfault.
uname -a Linux 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
`bpython version 0.15 on top of Python 3.4.3 /usr/bin/python3

import ortools
ortools.version
'4.3.3805'
from ortools.constraint_solver import pywrapcp
solver = pywrapcp.Solver('my solver')
x = solver.IntVar(0, 3, 'var_name')
x is None
False
x == None
Segmentation fault (core dumped)`

Yes, I overloaded the == operator to support writing constraints.
I need to look at it.

I have fixed it for the CP-SAT solver.


could not allocate memory when use copilot

Type: Bug
I am using vscode to connect remote gpu cluster. when i use copilot on this remote cluster, it will give "WebAssembly.Memory(): could not allocate memory". Howerver, when i use vscode on my local machine, or a remote machine but not a gpu cluster, copilot works fine. I think it may be some priviledge problem, but i can not found where to fix it
Extension version: 0.21.2
VS Code version: Code 1.94.2 (384ff7382de624fb94dbaf6da11977bba1ecd427, 2024-10-09T16:08:44.566Z)
OS version: Windows_NT x64 10.0.22631
Modes:
Remote OS version: Linux x64 4.18.0-80.7.1.el8_0.x86_64

System Info

Item
Value

CPUs
12th Gen Intel(R) Core(TM) i5-12490F (12 x 2995)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabledwebnn: disabled_off

Load (avg)
undefined

Memory (System)
31.85GB (16.72GB free)

Process Argv
--crash-reporter-id 110fa692-a640-41ab-b744-38c187be9dc0

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: A40

OS
Linux x64 4.18.0-80.7.1.el8_0.x86_64

CPUs
Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (40 x 1011)

Memory (System)
140.00GB (101.19GB free)

VM
0%

A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementsc:30995553
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
0ee40948:31013168
dvdeprecation:31068756
dwnewjupytercf:31046870
impr_priority:31102340
nativerepl1:31139838
refactort:31108082
pythonrstrctxt:31112756
wkspc-onlycs-t:31132770
wkspc-ranged-t:31151552
cf971741:31144450
autoexpandse:31146404
iacca1:31156133
notype1:31157159
showchatpanel:31153267
5fd0e150:31155592
dwcopilotcf:31162479
iconenabled:31158251

I see this error message when I use the copilot chat box. When I ask it a question, it only reply little words(less than 10 words), which it's no completed, and then it return the error message.

by the way, I also find a similar situation in this link: https://github.com/orgs/community/discussions/45418, and I try the solution mentioned in here, it doesn't work.

I also noticed that Copilot works well when I use it to explain some code in .txt, .json format, etc. But for .py files, it will always give  WebAssembly.Memory(): could not allocate memory" error. I think maybe .py files need more memory to execute by Copilot, so it fail.

I found the issue. I am using a remote server cluster with a SLURM system. When I use VSCode SSH to connect to the user login node, the problem occurs because the login node does not have GPU resources allocated (and possibly other resources). However, if I use salloc to request a GPU node and then SSH into the GPU node on the cluster, the issue does not occur.


Fix resources for builder-base presubmits

Fixes issue with pod not starting on Fargate.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


Support dark mode for graph view

Thank @zubayrrr for pointing this out in issue #14 .
We should definitely improve the graph view in dark mode.
Today is monday and my daily work is occupying, so I may need another weekend for solve this.

Fixed in v0.7.4


Add sad Akemi to cannot connect page

The current error page features a very ugly drawing. I think sad Akemi would be better suited.

Added Sad Akemi to the error page with the new Dark/Light themes.


Split UnaryOps.CAST into CAST and BITCAST

This PR introduces UnaryOps.BITCAST, and it is used instead of UnaryOps.CAST and bitcast=true boolean flag combination.
a = Tensor([1,2,3], dtype=dtypes.float).bitcast(dtypes.int).realize()
Before
  0 ━┳ STORE MemBuffer(idx=0, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))
  1  ┗━┳ CAST (dtypes.int, True)
  2    ┗━━ LOAD MemBuffer(idx=1, dtype=dtypes.float, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))
((LazyOp(op=BufferOps.STORE, src=(LazyOp(op=UnaryOps.CAST, src=(LazyOp(op=BufferOps.LOAD, src=(), arg=MemBuffer(idx=1, dtype=dtypes.float, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))),), arg=(dtypes.int, True)),), arg=MemBuffer(idx=0, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))),), [Opt(op=OptOps.LOCAL, axis=0, amt=3)])

After
 0 ━┳ STORE MemBuffer(idx=0, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))
  1  ┗━┳ BITCAST dtypes.int
  2    ┗━━ LOAD MemBuffer(idx=1, dtype=dtypes.float, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))
((LazyOp(op=BufferOps.STORE, src=(LazyOp(op=UnaryOps.BITCAST, src=(LazyOp(op=BufferOps.LOAD, src=(), arg=MemBuffer(idx=1, dtype=dtypes.float, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))),), arg=dtypes.int),), arg=MemBuffer(idx=0, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),)))),), [Opt(op=OptOps.LOCAL, axis=0, amt=3)])

Good to see you back!

arg of cast should be changed to only be the dtype, we don't need the True/False anymore since CAST/BITCAST are different ops

arg for casts is now just the dtype. Two failing tests are due to old sops.gz, i’ll update it.

@Qazalin @geohot

arg is now just the dtype (it's not a tuple anymore)
no bitcast on ImageDType
regenerated dataset so that sops.gz has the new UnaryOps.CAST and UnaryOps.BITCAST in the ASTs (for this I also had to update the UNROLL action in search, because I got a wrong action for UNROLL, axis=4 for the new dataset)

nice - new graph:

I'm not sure about the wrong UNROLL action though - What was the error?

I was getting this for PYTHONPATH="." GPU=1 python3 extra/optimization/get_action_space.py:

did you try bumping the DB version for the actions issue? you might just need to revalidate the sqlite db instead of changing action space script.

Still wrong after db bump, only seems to be solved by bumping axis in unroll.

the error is saying the handcoded opt had used a action that's not in the search action lists. can you print the kernel that caused this?

congrats<EMAIL_ADDRESS>to claim


Use HDBSQL as a SQL query tool with SAP HANA, express edition

Hi!
It is worth to mention that usr/sap folder is not created on macOS due to Apple System Integrity Protection https://support.apple.com/en-us/HT204899
So mac users will use their local folders for hdbclient tools.

Hi @vsenn,
In step, it is mentioned that the path might be different from the one listed after, as users are free (or constrained) to use a different folder.
I'm happy to list the default value for mac, but the support article reference is probably more appropriate in the following tutorial: Installing SAP HANA HDB Client (Mac).
Best Regards


Allow non-strict email address validation

Add ability to specify whether EmailAddressValidator used by AddressList should validate the email with the strict flag set or not.

This would be very useful. I'm using Zend-Mail to parse emails that arrive in an inbox and therefore have no control over their header values. When calling Zend\Mail\Storage\Imap->getMessage() an Exception is thrown from deep inside the Zend Framework if the Return-Path header contains a long address:
Fatal error: Uncaught Zend\Mail\Exception\InvalidArgumentException: The input exceeds the allowed length in vendor/zendframework/zend-mail/src/Address.php:41
Stack trace:
#0 vendor/zendframework/zend-mail/src/AddressList.php(249): Zend\Mail\Address->__construct('...', NULL)
#1 vendor/zendframework/zend-mail/src/AddressList.php(35): Zend\Mail\AddressList->createAddress('...', NULL)
#2 vendor/zendframework/zend-mail/src/AddressList.php(113): Zend\Mail\AddressList->add('...', NULL)
#3 vendor/zendframework/zend-mail/src/Header/AbstractAddressList.php(87): Zend\Mail\AddressList->addFromString('...')
#4 vendor/zendframework/zend-mail/src/Headers.php(482): Zend\Mail\Header\AbstractAddressList::fromString('To: ...')
#5 vendor/zendframework/zend-mail/src/Headers.php(231): Zend\Mail\Headers->loadHeader('To: ...')
#6 vendor/zendframework/zend-mail/src/Headers.php(95): Zend\Mail\Headers->addHeaderLine('To: ...')
#7 vendor/zendframework/zend-mime/src/Decode.php(142): Zend\Mail\Headers::fromString('Return-Path: ...', '\n')
#8 vendor/zendframework/zend-mail/src/Storage/Part.php(112): Zend\Mime\Decode::splitMessage('Return-Path: ...', 'Return-Path: ...', '')
#9 vendor/zendframework/zend-mail/src/Storage/Message.php(54): Zend\Mail\Storage\Part->__construct(Array)
#10 vendor/zendframework/zend-mail/src/Storage/Imap.php(128): Zend\Mail\Storage\Message->__construct(Array)
#11 mycode.php(31): Zend\Mail\Storage\Imap->getMessage(18)

A quick and dirty solution would be to add some static method like setStrictMode to the EmailAddress class. Any better ideas?

I completely agree with @steffenweber. We use Zend Mail along with Zend Mime to parse bounce reports. And those often contain something like Return-Path: <> which triggers exception in AddressList.

+1

Similar issue here. When iterating through an inbox (where you don't have control over the messages in there), I am e.g. running into Uncaught Zend\Mail\Exception\InvalidArgumentException: 'example.org ()' is not a valid hostname for the email address in zendframework/zend-mail/src/[Address.php:77](https://github.com/zendframework/zend-mail/blob/master/src/Address.php#L77).
In my case the To header contains this garbage: To<EMAIL_ADDRESS>().
Is there a good way to work around this? I'd guess that almost every "real life" inbox contains mails with such unexpected headers, which makes it almost impossible to iterate over an inbox.


Allow dream2nix to consume itself

Dream2nix should be able to consume itself.
What does that mean?
Dependencies packaged with dream2nix should be able to import as flake in a consuming d2n project.
Also, Sub-dependencies could be imported by dependencies with the same mechanism.
Currently
project/flake.nix

{
  inputs = {
    dream2nix.url = "github:nix-community/dream2nix";
    dep1.url = "github:org/name";
    dep1.flake = false;
  };
  outputs = inp:
    inp.dream2nix.lib.makeFlakeOutputs {
      systemsFromFile = ./nix_systems;
      config.projectRoot = ./.;
      source = ./.;
<EMAIL_ADDRESS>= {
        src = (import inp.dep1 {system = "x86_64-linux";}).module;
      };
    };
}

where module is a valid node_module
It is also needed to override the installPhase (of the library) as d2n does not output a valid node_module.
Then manually add the scr override to the consuming package (see above).

The provided flake is already fairly short. I'm not sure how we can optimize that further.
Should we just add an example to the docs?

We could remove that prefix and install the package directly in $out

That would allow for directly using an output as an input. For that to work the output must follow the npm conventions. (At least a valid package.json with a main entry. See also here
npm official describes its package format:

A package is any of the following:
a) A folder containing a program described by a package.json file.
b) A gzipped tarball containing (a).
c) A URL that resolves to (b).
d) A @ that is published on the registry with (c).
e) A @ that points to (d).
f) A  that has a latest tag satisfying (e).
g) A git url that, when cloned, results in (a).

I think that could be the default output of a package. (-> point a) )
Still we should investigate how other use-cases (electron apps, node-gyp, bins etc are related to this change)

Still we should investigate how other use-cases (electron apps, node-gyp, bins etc are related to this change)

I don't see a reason why it should have an effect on any of those. Those tools do not consume our outputs directly. There is a layer in between which symlinks/copies those dependencies together. That layer needs to be changed.
Currently the dependency linking/copying logic implemented in python depends on the current structure and would need to be adapted.
... or re-written.

Maybe this could be an RFC:
The output of a subsystem build should be a valid package of the same subsystem.
So the output of any dream2nix build can be consumed as a ready-built package (as long as you are in the same subsystem)
does this make sense?

The problem is, that consumable packages must be shipped without dependencies and applications should come with dependencies. Sometime we need both of these.
We could just generate two outputs. For example we could have out containing the installed output with dependencies and lib containing the output without dependencies. Then, whenever you need the package as dependency, you consume {package}.lib, otherwise {package}.out (or just {package}).


[Question]: Unstable Version Detected

What is your question?
Hi! I would like to use these plugins

connectivity_plus
device_info_plus
package_info_plus

in my Flutter application.
My CI/CD procedure checks for any vulnerable dependencies. For these plugins, an unstable version is detected due to the version indicated in the build.gradle file of the native Android component (1.0-SNAPSHOT). Is it possible to align the version in the build.gradle file with the same version as the Dart side plugin?
Checklist before submitting a question

[X] I searched issues in this repository and couldn't find such bug/problem
[X] I Google'd a solution and I couldn't find it
[X] I searched on StackOverflow for a solution and I couldn't find it
[X] I read the README.md file of the plugin
[X] I am using the latest version of the plugin
[X] All dependencies are up to date with flutter pub upgrade
[X] I did a flutter clean
[X] I tried running the example project

mhh, I see what you mean. I think simply changing version '1.0-SNAPSHOT' to version '1.0' could be enough.
There is probably a lint option to disable that check as well.
The packages in flutter/packages have the same thing too:
https://github.com/flutter/packages/blob/e21f1ee67c0db17fe4d5587040c62d8da59e54e5/packages/url_launcher/url_launcher_android/android/build.gradle#L2C1-L2C23
Do you get warnings from their packages too?

Yes, I think the dependency analysis tool gets suspicious about the word "snapshot".
I have the same problem with these dependencies from flutter/packages:

path_provider_android
url_launcher_android
I just opened a similar issue on their repository.

Got you, please post the link to the ticket here as well, I'd like to know what they decide

Sure, this is the related issue on flutter/packages: https://github.com/flutter/flutter/issues/156394

I got this error when running on android. is it the same issue? this issue happened when I update to latest version flutter
[  +97 ms] FAILURE: Build failed with an exception.
[        ] * What went wrong:
[        ] Execution failed for task ':device_info_plus:compileDebugJavaWithJavac'.
[        ] > Could not resolve all files for configuration ':device_info_plus:androidJdkImage'.
[        ]    > Failed to transform core-for-system-modules.jar to match attributes {artifactType=_internal_android_jdk_image, org.gradle.libraryelements=jar, org.gradle.usage=java-runtime}.
[        ]       > Execution failed for JdkImageTransform:

my flutter doctor:
[✓] Flutter (Channel stable, 3.24.3, on macOS 15.0.1
    24A348 darwin-arm64, locale en-MY)
[✓] Android toolchain - develop for Android devices
    (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2024.2)
[✓] VS Code (version 1.92.2)
[✓] VS Code (version 1.93.0-insider)
[✓] Connected device (6 available)
[✓] Network resources


⚠ Secret Website 3 has degraded performance

In 526beb6, Secret Website 3 ($SECRET_WEBSITE_3) experienced degraded performance:

HTTP code: 200
Response time: 2578 ms

Resolved: Secret Website 3 performance has improved in a460b10.


prof2 shows call path within GPU kernels in trace when it shouldn't

When measuring and analyzing LAMMPS with prof2 in the johnmc branch, I found that prof2 is adding detail about call paths internal to the GPU to the heterogeneous call chain for a kernel call in the GPU trace. We don't trace information within GPU kernels so we can't display it.
The figure below illustrates the problem. Notice that the trace shows a deep call path WITHIN the GPU kernel so that all of the elements in the trace appear to be in <unknown procedure> (green at the bottom of the call chain) rather than in Kokkos::impl::cuda_parallel_launch_constant_memory<...>() (the item on the path immediately below <gpu kernel>. Even when PC sampling, we don't trace internal details of kernels because we get GPU PC samples as a histogram (with the time dimension collapsed out).

The database that yielded this figure can be found in ufront.cs.rice.edu:~johnmc/bugs/prof2-trace-bug.tgz

These parts of the trace refer to the IP d0cb001a8cc42528f80012cf1143294c.gpubin+0x8f880, which the Structfile maps to a long inlined call stack ending in <unknown procedure>:
<P i="623" n="Kokkos::Impl::cuda_parallel_launch_constant_memory<Kokkos::Impl::ParallelReduce<LAMMPS_NS::ComputeTempKokkos<Kokkos::Cuda>, Kokkos::RangePolicy<Kokkos::Cuda, LAMMPS_NS::TagComputeTempScalar<0> >, Kokkos::InvalidType, Kokkos::Cuda> >() [d0cb001a8cc42528f80012cf1143294c.gpubin]" ln="_ZN6Kokkos4Impl36cuda_parallel_launch_constant_memoryINS0_14ParallelReduceIN9LAMMPS_NS17ComputeTempKokkosINS_4CudaEEENS_11RangePolicyIJS5_NS3_20TagComputeTempScalarILi0EEEEEENS_11InvalidTypeES5_EEEEvv" s="17" l="102" v="{[0x8f880-0x8f881)}">

                                            <S i="645" l="102" v="{[0x8f880-0x8f890)}" />

So the <unknown procedure> part is a case of "garbage in, garbage out."
Otherwise, this is very similar to 1d7b1a38a31a55f8d8786e3a368f6dec166d279d (couldn't find the PR for it). Based on that, possible solutions:

Add a flag to the trace sample structure to indicate that the sample refers to the GPU kernel (i.e. kernel-launch-edge) instead of the instruction itself. Adding one flag byte will increase the size of a measurement trace by 8%, two by 16%, etc.
Add a flag to the trace header to indicate that all samples in the trace refer to the GPU kernel instead of the instructions themselves. This may require some care in Run to ensure edge-traces are not mixed with instruction-traces.
Add logic in Prof2 to consider trace-points just below a <gpu kernel> placeholder as referring to the GPU kernel instead of the instruction itself. This will make adding "real" GPU traces later much harder (and add unpreferred logic "magic" to Prof2).

I think the solution is the last:

Add logic in Prof2 to consider trace-points just below a  placeholder as referring to the GPU kernel instead of the instruction itself. This will make adding "real" GPU traces later much harder (and add unpreferred logic "magic" to Prof2).

We will NEVER have traces at a finer granularity than instructions from GPUs.  At best, we will receive a sequence of snapshots, where a snapshot includes each active or "on deck" PC across the entire GPU. Such a snapshot would indicate where every wave/warp/workgroup-item-set is at a point in time across all the compute units/execution units/streaming multiprocessors. That is basically a histogram at an instant of time. We can't build a finer-grain trace out of those.
The only way we could get something finer grained is with instrumentation. I don't think it is worth the cost.

After looking at this a bit more, another alternative is to just use a filter to make <unknown procedure> disappear from the bottom of that call chain in the kernel. The rest of the chain indicates where the first instruction came from.
However, the first instruction isn't the only instruction in the kernel. I think focusing on the call chain for the first instruction gives an illusion of precision when there is none.

After looking at this a bit more, another alternative is to just use a filter to make  disappear from the bottom of that call chain in the kernel. The rest of the chain indicates where the first instruction came from.

The <unknown procedure> is a bug in either Struct or NVIDIA's DWARF, I think the innermost inlined call should probably be a single-alien (i.e. roughly a macro expansion) instead of a double-alien. So I wouldn't go making changes to the Viewer when the bug is elsewhere (we've done that before, but I'd like to keep it limited to necessary evils).

However, the first instruction isn't the only instruction in the kernel. I think focusing on the call chain for the first instruction gives an illusion of precision when there is none.

Completely agreed.

We will NEVER have traces at a finer granularity than instructions from GPUs. At best, we will receive a sequence of snapshots, where a snapshot includes each active or "on deck" PC across the entire GPU. Such a snapshot would indicate where every wave/warp/workgroup-item-set is at a point in time across all the compute units/execution units/streaming multiprocessors. That is basically a histogram at an instant of time. We can't build a finer-grain trace out of those.

Couldn't we in theory build an instruction trace per workgroup/SM/CU/... using those snapshots? It might not be worth it now, but given the amount of interest in hardware-level performance analysis IMHO we shouldn't discount the possibility this early in the game. (Also I want to avoid adding magic logic in Prof2 if possible, it's a bear to fix when assumptions shift and the magic flops.)
So IMHO, the second option seems the most appropriate, it has minimal cost now and won't cause headaches later.

So IMHO, the second option seems the most appropriate, it has minimal cost now and won't cause headaches later.

We can certainly add a flag to the trace header.

Couldn't we in theory build an instruction trace per workgroup/SM/CU/... using those snapshots? It might not be worth it now, but given the amount of interest in hardware-level performance analysis IMHO we shouldn't discount the possibility this early in the game. (Also I want to avoid adding magic logic in Prof2 if possible, it's a bear to fix when assumptions shift and the magic flops.)

That could be very useful for large kernels.
However, it is super expensive, which could cause hundreds of overhead with gigabytes of measurement data.

However, it is super expensive, which could cause hundreds of overhead with gigabytes of measurement data.

I would think this would generate roughly the same data volume as ~8-16 CPU threads, assuming the snapshots happen on a regular interval (e.g. 500us). That's well within what Prof2 (should) handle.

resolved by #622

@blue42u I think prof2 can handle thousands of threads without any problem. On the CU/SM level, it would be equivalent of >100 CPU threads per GPU, considering A100 with 108 SMs; on the scheduler level, it could be > 400 CPU threads. However, I was worried about the runtime (hpcrun) overhead but not postmortem analysis overhead, taking a snapshot or copying large amount of data from the GPU to the CPU is expensive.