Error in implementing Custom ResNet50 model

I am trying to deploy my ResNet-50 custom trained classification model using pytorch lite library. The model works well on python but giving false predictions on Flutter, always giving high probs for the first class.
I optimized my model using this code provided in the Readme
from torch.utils.mobile_optimizer import optimize_for_mobile
best_model = best_model.to('cpu')

best_model.eval()
example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(best_model, example)
optimized_traced_model = optimize_for_mobile(traced_script_module)
optimized_traced_model._save_for_lite_interpreter("Fitrzpack_model.pt")

I don't know the exact reason but may this is due to the transformation I have to apply to my images during prediction or something else. I have also provided the transformation code below.
transform = transforms.Compose([transforms.ToPILImage(),
                               transforms.ToTensor(),
                               transforms.Resize((IMG_SIZE, IMG_SIZE)),
                               transforms.Normalize(mean=[0.485, 0.456, 0.406],
                     std=[0.229, 0.224, 0.225])
                     ])

I am also sharing the code I use to predict the model in python
def predict(x):
    img = Image.open(x).convert("RGB")
    img = transform(np.array(img))
    print("shape of transformed image", img.shape)
    img = img.view(1, 3, 224, 224)
    print("shape of reshaped image", img.shape)
    best_model.eval()
    with torch.no_grad():
        if torch.cuda.is_available():
            img = img.cuda()

        out = best_model(img)

        return out.argmax(1).item()

Please help me getting out of this situation @abdelaziz-mahdy

Most probably it's a problem with the input sooo please give both your full prediction code for dart and Python

also does it have the same results on android and ios?

Hi @abdelaziz-mahdy, Thank you so much for your response.
This is how I am loading the model
String pathImageModel =
        "assets/models/best_Fitrzpack_withNorm_extraTransformations_Adam_epoch_22.pt";
    try {
      _imageModel = await PytorchLite.loadClassificationModel(
          pathImageModel, 224, 224, 5,
          labelPath: "assets/labels/label_classification_imageNet.txt");

And this is this how I am making prediction in Dart, I also tried with default mean and std but results were similar.
textToShow = await _imageModel!.getImagePrediction(
        await File(image.path).readAsBytes(),
        mean: [0.485, 0.456, 0.406],
        std: [0.229, 0.224, 0.225]);
    textToShow = "${textToShow ?? ""}, ${inferenceTimeAsString(stopwatch)}";

Here you can also see the output of the prediction list, the first value is always higher hence always prediction first class in every image
[37.45313262939453, 20.606271743774414, 19.992067337036133, 9.70131778717041, 32.07078552246094, -79.898193359375, -94.40465545654297, -99.96098327636719, -95.91107940673828, -102.2623291015625, -94.18521881103516, -104.00025939941406, -97.89167022705078, -102.9541244506836, -101.01667785644531, -101.24066925048828, -103.78877258300781, -96.97942352294922, -94.73454284667969, -104.4057388305664, -98.69512939453125, -96.98098754882812, -95.92950439453125, -99.71725463867188, -36.406681060791016, -95.3742446899414, -93.91866302490234, -93.91046142578125, -97.62044525146484, -78.75592803955078, -96.0226821899414, -100.22040557861328, -108.44017791748047, -81.2840805053711, -96.94139862060547, -92.8824234008789, -102.83641815185547, -104.82340240478516, -91.39096069335938, -103.66374206542969, -104.43321990966797, -106.07219696044922, -106.11819458007812, -91.6676254272461, -99.1351089477539, -102.29850006103516, -99.13842010498047, -90.38143157958984, -106.93729400634766, -90.3895263671875, -93.90796661376953,

Python codes
Below is the python code used for model loading and export

# Model innitialization
resnet = resnet50(weights=ResNet50_Weights.IMAGENET1K_V2)
num_ftrs = resnet.fc.in_features
resnet.fc.in_features = nn.Linear(num_ftrs, OUT_CLASSES)

model = deepcopy(resnet)

# Custom trained model loading
model_path ="models/pytorch_skin_types/best_Fitrzpack_withNorm_extraTransformations_Adam_epoch_22.pth"
best_model = model
best_model.load_state_dict(torch.load(model_path,weights_only=True))
best_model.eval()

# Export code used to optimize the model
from torch.utils.mobile_optimizer import optimize_for_mobile
best_model = best_model.to('cpu')

best_model.eval()
example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(best_model, example)
optimized_traced_model = optimize_for_mobile(traced_script_module)
optimized_traced_model._save_for_lite_interpreter("Fitrzpack_model.pt")

Below is the prediction function used in python
label_index = {"Phototype_I_&_II": 0, "Phototype_III": 1, "Phototype_IV": 2, "Phototype_V": 3, "Phototype_VI": 4}
index_label = {0 :"Phototype_I_&_II", 1 : "Phototype_III", 2 : "Phototype_IV", 3 : "Phototype_V", 4 : "Phototype_VI"}

# Transformations
transform = transforms.Compose([transforms.ToPILImage(),
                               transforms.ToTensor(),
                               transforms.Resize((IMG_SIZE, IMG_SIZE)),
                               transforms.Normalize(mean=[0.485, 0.456, 0.406],
                     std=[0.229, 0.224, 0.225])
                     ])

# prediction function
def predict(x):
    img = Image.open(x).convert("RGB")
    img = transform(np.array(img))
    print("shape of transformed image", img.shape)
    img = img.view(1, 3, 224, 224)
    print("shape of reshaped image", img.shape)
    best_model.eval()
    with torch.no_grad():
        if torch.cuda.is_available():
            img = img.cuda()

        out = best_model(img)

        return out.argmax(1).item()

print(index_label[predict(r"D:\262541-Faizan\misc_projects\face_analyzer\Dataset\Fitrzpack_test\black.jpg")])
print(index_label[predict(r"D:\262541-Faizan\misc_projects\face_analyzer\Dataset\Fitrzpack_test\white.jpg")])

If possible can you provide the model and a test image try it out and make sure whatever fix I do works

https://drive.google.com/drive/folders/1ncrauuM5S1VcSIDu485kzQgX46RdOH_P?usp=sharing
Sure , you can get the model and the training notebook in above drive folder.

Will check it out, I can't promise I will be able to figure it out or not but will try it

Yeah, sure
Have you ever deployed ResNet50 using this library ?
You can also share that if possible

Me no, but any classification model should work correctly, since it's just using pytorch
The only problem is the input has to be the same as the python one so
same format RGB
Same normalization
Same image encoding I think too
So alot of input parameters to do

Also did you try the other preprocessing enum?

I just tried with Standard mean and Std dev of ImageNet which I also used for in python. I also tried what th default parameters, but it always predicts the first class

transform = transforms.Compose([transforms.ToPILImage(),
                               transforms.ToTensor(),
                               transforms.Resize((IMG_SIZE, IMG_SIZE)),
                               transforms.Normalize(mean=[0.485, 0.456, 0.406],
                     std=[0.229, 0.224, 0.225])
                     ])

considering this python code for transformations, how would you suggest the prediction function to be called ?
Another concern is that my model is not converting into scripted_model.pt properly.

The transformation mentioned is the same as the one that package does
I don't know if not converting will hurt or not I only tested with converted models

ok so i went through both python code and dart code (i cant try it since i dont have the labels for the test images)
did you try using
    List<double?>? predictionList = await _imageModel!.getImagePredictionList(
      await File(image.path).readAsBytes(),
      preProcessingMethod: PreProcessingMethod.imageLib
    );

and
    List<double?>? predictionList = await _imageModel!.getImagePredictionList(
      await File(image.path).readAsBytes(),
      preProcessingMethod: PreProcessingMethod.native
    );

did both give the same results? since from i am currently seeing everything should work correctly
also can you load the model file in python and try it to make sure the downloaded model is correct?

Yes, the model works well in python. I have checked it multiple times.I have also provided the code and model in the above link. you can also check. labels are also there

The test images i provided is random. you just need to check if its predicting anything other than the first class.

I don't know the correct label for them, so that will not help😅

I skipped the optimized_traced_model = optimize_for_mobile(traced_script_module) line and it worked somehow.
I also wanted to know how can we run YOLOv8 classification models using Flutter pytorch lite library. I tried with the same torchscript method used for YOLOv8 object detection but it doesn't worked out.

I am glad that your model worked, I was stuck not figuring out why it was failing to work
Sorry for not being to help
For yolov8 did you create it using the yolo command provided in the readme?

Thanks a lot for your help, I used this code, but its not supported
model.export(format="", imgsz = 320, optimize = False)

Dont know how we can run YOLOv8 classifiation models

What do you mean by not supported? It doesn't work? And if there is an error please share it

There was an error in model loading in Flutter. May be we need to convert YOLO classification model differently than object detection model. I used the code below which is obviously for YOLOv8 object detection but the conversion method for YOLOv8 classification model is not explicitly mentioned in the Readme.
!yolo mode=export model="your model" format=torchscript optimize

did you try model.export(format="torchscript", imgsz = 320, optimize = True)

@MeDenTec did you try the above export? if yes and it works please let me know to close this issue

model.export(format="torchscript", imgsz = 320, optimize = True)
Yes, I tried the similar export method, but it didn't worked. I thinks its only for Object detection models not YOLO image classification models.
Don't know how can I run yolo image classification models.

It's in the official docs of yolov8 so this weird
What didn't work exactly? What were the errors ?
Please provide as much info as possible since I can't fix something I don't know 😅

Since I haven’t heard back from you in a while, I’ll assume everything worked as expected. If you encounter any issues, feel free to let me know!


Implement the new precedence rules for resources_dir for dapr run -f

Describe the proposal

Implement the new precedence rules for resources_dir for dapr run -f

Add capability to look for resources under given input directory and fallback to default if not present with UTs.
See the folder organization structure  in dapr/proposals#6.
related to #1123
Release Note

RELEASE NOTE:

/assign


wizard: WAN over PPPOE - default Route is not set correctly

Version
OPNsense 18.1-amd64
FreeBSD 11.1-RELEASE-p6
OpenSSL 1.0.2n 7 Dec 2017
Failure history

16.7.3: Fails
17.1.4: Fails
17.7: Fails
18.1: Fails
18.7 Fails

Background
The default route is set to go the physical WAN port, rather than the PPPoE virtual port:

I did disconnect the PPPoE connection, so the WAN interface was down; and then connected it again
I did re-save the WAN configuration; thinking (hoping) that it simply would fix itself

Steps to reproduce
Start fresh, and run the configuration wizard
Welcome to OPNSense! Starting wizard.

Next

General information.

Next

Time Server Information.

Timezone: America/Toronto
Next

Configure WAN Interface

Selected Type: PPPoE
PPPoE Username: [redacted]
PPPoE Password: [redacted]
Next

Configure LAN Interface

Next

Set Admin Web GUI Password

Admin Password: [redacted]
Admin Password AGAIN: [redacted]
Next

Reload Configuration

Reload

Finished initial configuration!

Congratulations! OPNsense is now configured.
Click to continue to the dashboard.

You are currently running in LiveCD mode. A reboot will reset the configuration. SSH remote login is disabled.

How are the routes
So now it is setup. Lets check the routes
System -> Routes -> Status

Proto
Destination
Gateway
Flags
Use
MTU
Netif
Netif (name)

ipv4
Default
link#2
U
5083
1500
re0

ipv4
[ISP gateway IP redacted]
link#7
UH
0
1492
pppoe
wan

So it's using the wrong interface as the default route. Lets try pinging.
Interfaces -> Diagnostics -> Ping

Host: <IP_ADDRESS>

IP Protocol: IPv4

Source Address: Default

Count: 3
PING <IP_ADDRESS> (<IP_ADDRESS>): 56 data bytes

 --- <IP_ADDRESS> ping statistics ---
 3 packets transmitted, 0 packets received, 100.0% packet loss
 ping: sendto: Host is down
 ping: sendto: Host is down
 ping: sendto: Host is down

Fails. Lets force use of the WAN (i.e. pppoe0) interface:

Host: <IP_ADDRESS>

IP Protocol: IPv4

Source Address: WAN

Count: 3
PING <IP_ADDRESS> (<IP_ADDRESS>) from [redacted]: 56 data bytes
64 bytes from <IP_ADDRESS>: icmp_seq=0 ttl=51 time=24.503 ms
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=51 time=24.306 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=51 time=28.037 ms

--- <IP_ADDRESS> ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 24.306/25.615/28.037/1.714 ms

So if i force the use of the interface, the packets will go. It just doesn't realize the default route.
 What about pinging again
Interface -> Diagnostics -> Ping

Host: <IP_ADDRESS>

IP Protocol: IPv4

Source Address: Default

Count: 3
PING <IP_ADDRESS> (<IP_ADDRESS>): 56 data bytes

--- <IP_ADDRESS> ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down

Nope, still cannot ping using the default gateway.
Disable and re-enable the WAN interface
Lets disable the interface:
Interfaces -> [WAN]

Enable: [x] Enable Interface
Uncheck
Enable: [ ] Enable Interface
Save
Apply changes

And re-enable it:

Enable: [ ] Enable Interface
Check
Enable: [x] Enable Interface
Save
Apply changes

And check the routes again:
System -> Routes -> Status

Proto
Destination
Gateway
Flags
Use
MTU
Netif
Netif (name)

ipv4
Default
link#2
U
12152
1500
re0

ipv4
[ISP gateway IP]
link#7
UH
0
1492
pppoe
wan

Still the wrong route. And ping still fails:
Interfaces -> Diagnostics -> Ping

Host: <IP_ADDRESS>

IP Protocol: IPv4

Source Address: Default

Count: 3
PING <IP_ADDRESS> (<IP_ADDRESS>): 56 data bytes

--- <IP_ADDRESS> ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down

And again if i force the use of PPPoE interface directly, it works:

Host: <IP_ADDRESS>

IP Protocol: IPv4

Source Address: WAN

Count: 3
PING <IP_ADDRESS> (<IP_ADDRESS>) from [redacted]: 56 data bytes
64 bytes from <IP_ADDRESS>: icmp_seq=0 ttl=51 time=24.503 ms
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=51 time=24.306 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=51 time=28.037 ms

--- <IP_ADDRESS> ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 24.306/25.615/28.037/1.714 ms

Even though the PPPoE interface is up, the default interface is out the physical WAN port - with traffic going nowhere.
System -> Routes -> Status

Proto
Destination
Gateway
Flags
Use
MTU
Netif
Netif (name)
Expire

ipv4
default
link#2
U
81208
1500
re0

ipv4
[ISP gateway IP]
link#7
UH
0
1492
pppoe0
wan

Even though my PPPoE interface is up
System -> Gateways -> Status

Name
Gateway
Monitor
RTT
Loss
Status
Description

WAN_PPPOE
[ISP gateway IP]
none
0.0 ms
0.0 %
Online
Interface WAN_PPPOE Gateway

If you compare that to pfSense:

Destination
Gateway
Flags
Use
Mtu
Netif
Expire

default
[ISP gateway IP]
UGS
48915
1492
pppoe0

<IP_ADDRESS>/24
link#1
U
9999049
1500
em0

[ISP gateway IP]
link#7
UH
172033
1492
pppoe0

Brilliant report!
One point, though: note what has been redacted, then search those bits in your report before submitting it :) - looking at that address in block <IP_ADDRESS>/18.

I'm having exactly the same issue after a fresh install of 18.1 (nano) with a PPPoE connection.
Replication and symptoms are the same.

Same on 18.1.4? If you want I can build a Nano image based on the latest code.

Possibly a silly question but: Where can I get 18.1.4? The downloads I'm using are labelled just 18.1.
I originally had most things working fine until I discovered port forwarding wasn't working properly, so I did all the updates that were available followed by a reboot, but the system didn't come up after that. Then I tried a fresh install (of 18.1) and encountered this problem.
If you can build a Nano with the latest code that would be great - thanks!

We don't have images for all 18.1.x releases as time to do QA is traded for pushing weekly updates. They are, however, sporadically released o fix image-based issues. :)
I'll provide a 18.1.4 Nano in a bit. Amd64 I guess?

Yes, Amd64 please. Thank you for taking the time to do this.

Sure, no problem. Here it is: https://pkg.opnsense.org/FreeBSD:11:amd64/snapshots/OPNsense-201803131410-LibreSSL-nano-amd64.img.bz2

Just to update on this - I'm now on OPNsense 18.1.5 and the problem has been resolved.
It's possible to upgrade from versions affected with the problem by manually adjusting the default route via the shell. I can't remember the exact commands I used now as it was a couple of weeks ago but it was along the lines of:
route delete default
route add default [gateway address]
For the gateway address - it was either my pppoe address, or the upstream gateway I used.
With the default gateway changed, it's then possible to upgrade and then the above commands aren't needed again :)

Looks like it was related to this fix: https://github.com/opnsense/core/commit/f4c5e21047 more or less relating to these to changes in 18.1.4 https://github.com/opnsense/changelog/blob/9f33a2a0623addbb7378e568893af032b8ffd0db/doc/18.1/18.1.4#L10-L11
We'll issue new images based on the upcoming 18.1.6 to replace the current 18.1 ones.
Please close if you feel this is solved completely. :)
Thanks,
Franco

@JackTrapper sorry for the delay, here's a VGA based on the upcoming 18.1.6 https://pkg.opnsense.org/FreeBSD:11:amd64/snapshots/OPNsense-201804070632-OpenSSL-vga-amd64.img.bz2

Same problem.  A lot of messages were added since 18.1 to help debug.

On 8. Apr 2018, at 16:47, JackTrapper<EMAIL_ADDRESS>wrote:
Same problem.
Reply to this email directly, view it on GitHub, or mute the thread.

System: Log File. Search for "ROUTING" in the input field.

After initial setup:
23:54:20: /rc.reload_all: ROUTING: setting IPv4 default route to [Gateway IP]
23:54:20: /rc.reload_all: ROUTING: no IPv6 default gateway set, trying wan on 'pppoe0' ()
23:54:20: /rc.reload_all: ROUTING: no IPv4 default gateway set, trying wan on 'pppoe0' ([Gateway IP])
23:54:20: /rc.reload_all: ROUTING: entering configure using defaults
23:54:17: /rc.reload_all: ROUTING: skipping IPv4 default route to wan
23:54:17: /rc.reload_all: ROUTING: skipping IPv4 default route to wan
23:54:17: /rc.reload_all: ROUTING: no IPv6 default gateway set, trying wan on 'pppoe0' ()
23:54:17: /rc.reload_all: ROUTING: no IPv4 default gateway set, trying wan on 'pppoe0' ([Gateway IP])
23:54:17: /rc.reload_all: ROUTING: entering configure using 'lan'
23:54:15: /rc.newwanip:   ROUTING: setting IPv4 default route to [Gateway IP]
23:54:15: /rc.newwanip:   ROUTING: no IPv6 default gateway set, trying wan on 'pppoe0' ()
23:54:15: /rc.newwanip:   ROUTING: no IPv4 default gateway set, trying wan on 'pppoe0' ([Gateway IP])
23:54:15: /rc.newwanip:   ROUTING: entering configure using 'wan'
23:54:14: /rc.reload_all: ROUTING: setting IPv4 default route to [Gateway IP]
23:54:14: /rc.reload_all: ROUTING: no IPv6 default gateway set, trying wan on 'pppoe0' ()
23:54:14: /rc.reload_all: ROUTING: no IPv4 default gateway set, trying wan on 'pppoe0' ([Gateway IP])
23:54:14: /rc.reload_all: ROUTING: entering configure using 'wan'
23:52:38: /rc.bootup:     ROUTING: no IPv6 default gateway set, trying wan on 're0' ()
23:52:38: /rc.bootup:     ROUTING: no IPv4 default gateway set, trying wan on 're0' ()
23:52:38: /rc.bootup:     ROUTING: entering configure using defaults

Disconnected and Re-connecting the WAN interface:
23:57:01: /rc.newwanip:   ROUTING: setting IPv4 default route to [Gateway IP]
23:57:01: /rc.newwanip:   ROUTING: no IPv6 default gateway set, trying wan on 'pppoe0' ()
23:57:01: /rc.newwanip:   ROUTING: no IPv4 default gateway set, trying wan on 'pppoe0' ([Gateway IP])
23:57:01: /rc.newwanip:   ROUTING: entering configure using 'wan'

It looks normal, unfortunately, always using pppoe0 except for rc.bootup, which is prior to wizard use although you probably configured re0 as WAN instead of using the automatic default re1? I'm not sure if that changes things. And speaking of the wizard, is that a general issue with PPPoE setup on your end or just the wizard misbehaving? What happens after a persistent reboot?
There needs to be a sort of baseline to when it works and when it doesn't and we don't have that except that although there are problems with PPPoE it should work better than this in any case. It looks like a transient bug and I focused on operational issues...
Before you enter the wizard, would you mind deleting the following file?
# rm /tmp/re0_defaultgw

Cheers,
Franco

you probably configured re0 as WAN instead of using the automatic default re1?

The machine has only two interfaces:

re0: onboard Realtek going to the modem (WAN)
em0: Intel Gigabit ethernet PCI card (LAN)

Usually that's how i (manually) configure them. This time i was too late pressing the keys, and it guessed right anyway. (In other words, i don't know what re1 would be, or why such a thing would come up).

And speaking of the wizard, is that a general issue with PPPoE setup on your end or just the wizard misbehaving?

As far as i know the Wizard executed properly. The PPPoE connection on the WAN adapter does connect and does function (if i force ping to use the PPPoE interface, i can ping Google, <IP_ADDRESS>, and everyone else on the Internet).

What happens after a persistent reboot?

This is all from a Live USB stick; i can't have it over-write my router's actual hard drive - because then i lose my current pfSense. :(   I never actually tried it before; but i always assume that a Live USB (like a Live CD) loses all it's configuration after reboot. Can it actually persistently run off a USB stick?

@JackTrapper yes, you can use a nano image for that, it is read-write after boot and configuration persists

(you can even use the nano to install your running configuration afterwards from the console, run "opnsense-installer")

Ok, the VGA image does not persist settings between reboots; the initial setup wizard starts.
I'll try a nano image tomorrow.

Where are you finding these latest snapshots from? The earlier links are gone.
Is there a place people should be going to find them?

All prior snapshot images are obsolete because of https://forum.opnsense.org/index.php?topic=7987.0 or see our main mirror for direct download https://pkg.opnsense.org/releases/18.1/

Noticed a strange thing with the Javascript, the subnet selection was not deactivated.
# opnsense-patch 076eb9f

This bug is still present in the official 18.1.6 release, nano abd dvd tested.
Starting from fresh USB image boot, set WAN and LAN interfaces manually or automatic, then using the pppoe wizard, the result is always the same: the internet doesn't work for the lan clients, but works on the router.
The routing table still displays re0 instead of pppoe0 as the default route.

Sure, it could be because it was fixed in 18.1.9, not 18.1.6.

Ok, that's great and sorry for not realizing this.
Is there any way to download a release version, newer than 18.1.9? Sorry for asking this here, I'm only finding 18.1.6.
I know that one can build the latest version from the sources, I'm just looking for an img for a newer version.

We don't usually do 18.1.x image releases, but if you let me know which image/arch you are looking for I can assemble one no problem. :)

If you could build an amd64 nano 18.1.9 or 18.1.10 img it would be great, thanks..

The issue is still present.
As i had done in the past, i was going to re-re-re document the issue for the current version.

i documented it in #850: issue was closed
i documented it in #2126; issue was closed

I tried to get an updated version; but was told they're gone.
So i was going to wait for a new version to come out so i could re-re-recreate the issue again.

@Crisr here you go https://pkg.opnsense.org/FreeBSD:11:amd64/snapshots/OPNsense-18.1.10-OpenSSL-nano-amd64.img.bz2
@JackTrapper All coding help, asking for new images or in-depth analysis is highly appreciated. The fix that went in after 18.1.6 is a likely candidate, but my hands are tied in verifying a working setup without having a PPPoE setup.

Thank you fichtner!

I have seen it say no default route set before but everything seems okay, if I could get an image for amd64 for a usb stick that can be installed on an A10 that would be great and much appreciated as always and I can help test as well ??.
The earliest I could have a look at will be next Monday though

From: Crisr<EMAIL_ADDRESS>Sent: Wednesday, June 27, 2018 9:10:51 PM
To: opnsense/core
Cc: Subscribed
Subject: Re: [opnsense/core] wizard: WAN over PPPOE - default Route is not set correctly (#2186)
Thank you fichtner!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/opnsense/core/issues/2186#issuecomment-400796662, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADddQsGFsOjMcCm_NKdHoa-Qu3w2KRaAks5uA9i7gaJpZM4SAwD0.

I'm clearly unable to fix this. Making room for somebody to step up.
@L1ghtn1ng image here https://pkg.opnsense.org/FreeBSD:11:amd64/snapshots/OPNsense-18.1.10-OpenSSL-serial-amd64.img.bz2

Thanks for the image, I have pppoe and even though I have seen this in the logs where it says it cannot set route it works fine, as I have stayed or seems to as I am none the wiser and get internet.
What I think would be a idea is to one under the ISP's in question that everyone is using. Getting packed traces of what is coming in to OPNsense and see what we get from comparing the packet dumps.
From: Crisr
Sent: Wednesday 27 June, 22:26
Subject: Re: [opnsense/core] wizard: WAN over PPPOE - default Route is not set correctly (#2186)
To: opnsense/core
Cc: J.Townsend, Comment
Well, the bug is still open in 18.1.10
No way to have a working opnsense router if you have a pppoe internet config using normal installation procedures. It could probably be fixed with some command line instructions but I don't have the knowledge to understand what I'm doing at that level.
It's kind of amazing that such a showstopper bug seems to be carried over so many builds with no fix, probably there are not many using pppoe configurations to connect to the ISPs
I've tried 2 installation procedures:
flashed the usb stick boot the image and then
Variant 1:
2a: set interfaces from command line
2b: set lan ip from command line
2c: access web interface and start setup wizard, setup pppoe config.
2d: access dashboard and check bot lan and wan are up an running (correct IPs loaded, traceroute working).
2e: bug is open: no route to lan clients, no internet access
Variant 2:
2a: wait for the boot to finish and make sure that LAN interface and opnsense box IP is the default <IP_ADDRESS> in the command line after boot
2b: access web interface and start setup wizard, setup pppoe config.
2c: access dashboard and check bot lan and wan are up an running (correct IPs loaded, traceroute working).
2d: bug is open: no route to lan clients, no internet access
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/opnsense/core/issues/2186#issuecomment-400817243, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADddQuWCBjRffMJ1f2cFXXmn082vUhMIks5uA-pVgaJpZM4SAwD0.

Update:
Well, it seems to be working after all.. don't know what caused to begin working, however I can list what steps I took after booting for the 1st time:

assigmed interfaces manually from the console options in opnsense (re0 is WAN in my config, opnsense wants to put re0 to LAN by default, so I switched the interfaces with the auto-config option, take out the cables and then put them back in)
set lan ip <IP_ADDRESS> from the opnsense console options.
Reboot. So far nothing special
On my LAN PC (win10) I've set the network card manually to a 192.168.1.x address to be able to reach opnsense web interface. Disabled and enabled the windows pc net interface and was able to enter the opnsense gui
Took the system wizard and setup pppoe credentials and other minor settings.
Checked the System: Routes: Status and the default route was set to re0 (my WAN) so the bug was still present at this stage (no internet available on the win10 pc). In the opnsense web dashboard, the pppoe connection was online and available
tried to add a new route in the configuration menu, <IP_ADDRESS>/24, wan_pppoe gateway. I didn't know what I was doing, it could be completely stupid, but I've tried it anyway, it didn't make any difference
from the opnsense command line I've tried 'route add -inet <IP_ADDRESS>/24 -link -iface pppoe0' but I've got a message that the route was already set..
removed the route I've added on step 7 from the web gui.
rebooted (I think, I'm not really sure about this)
on the windows 10 pc in the ip4 network adapter options, reverted to DHCP settings (this was changed on step 4).
enabled/disabled win10 network adapter on the win-10 pc.
and suddenly I've got internet and routes are fixed.
https://i.imgur.com/5tEeDiC.png

So yeah, don't know what has happened, don't know if it was something I did or did badly in the beginning, but so far it's working.
Hope this helps.

From what I understand this issue is about the wizard's way of setting up PPPoE which is reported to be wrong here, but this does not mean setup of PPPoE is wrong in general, which is part of why this is difficult to pin down without the actual setup and going through the trouble of inspecting what changed between wizard and manual steps.

Is it possible to rewrite the wizard to do the commands that we know to work when it's done manually? This in turn fix the issue once and for all?

From: Franco Fichtner<EMAIL_ADDRESS>Sent: Thursday, June 28, 2018 6:43:41 PM
To: opnsense/core
Cc: J.Townsend; Mention
Subject: Re: [opnsense/core] wizard: WAN over PPPOE - default Route is not set correctly (#2186)
From what I understand this issue is about the wizard's way of setting up PPPoE which is reported to be wrong here, but this does not mean setup of PPPoE is wrong in general, which is part of why this is difficult to pin down without the actual setup and going through the trouble of inspecting what changed between wizard and manual steps.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/opnsense/core/issues/2186#issuecomment-401099089, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADddQmW_DkUAH6kZeu6JdsNIMS1Nmrv3ks5uBQe9gaJpZM4SAwD0.

Has anyone tried 18.7 then?

Has anyone tried 18.7 then?

I downloaded 18.7 and I tried it this morning.
Same problem. The default route is configured to go out the physical WAN interface, rather than out the PPPoE interface:
OPNSense (System -> Routes -> Status)

Proto
Destination
Gateway
Flags
Use
MTU
Netif
Netif (name)
Expire

ipv4
default
link#2
U
423335
1500
re0

ipv4
[WAN IP]
link#7
UHS
0
16384
lo0

ipv4
[Gateway IP]
link#7
UH
279
1492
pppoe0
wan

ipv4
[DNS Server 1]
[Gateway IP]
UGHS
279
1492
pppoe0
wan

ipv4
[DNS Server 2]
[Gateway IP]
UGHS
0
1492
pppoe0
wan

ipv4
<IP_ADDRESS>/24
link#1
U
22872
1500
em0
lan

ipv4
<IP_ADDRESS>
link#1
UHS
0
16384
lo0

You can compare the equivalent from pfSense:
pfSense (Diagnosics -> Routes)

Destination
Gateway
Flags
Use
Mtu
Netif
Expire

default
[Gateway IP]
UGS
22004
1492
pppoe0

[WAN IP]
link#7
UHS
0
16384
lo0

[Gateway IP]
link#7
UH
363
1492
pppoe0

[DNS Server 1]
[Gateway IP]
UGHS
0
1492
pppoe0

[DNS Server 2]
[Gateway IP]
UGHS
0
1492
pppoe0

<IP_ADDRESS>/24
link#1
U
8297
1500
em0

<IP_ADDRESS>
link#1
UHS
0
16384
lo0

I realize this is all deep inside OpenBSD, and we have no idea what the OpenBSD networking subsystem is doing, or why. But is there some log file available through the WebGUI that would help?

This issue has been automatically timed-out (after 180 days of inactivity).
For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.


[Vulkan] Slow performance of GPU->CPU copy

My code takes results of rendering from a frame buffer to the CPU memory using an intermediate staging texture. I noticed that the frame rate I can get with the Vulkan backend on Windows is substantially (like 10 times) lower than with the Direct3D backend. Narrowing down the problem revealed the bottleneck is a code that copies from the mapped memory of a staging texture to the CPU memory. Which is actually a simple call to System.Buffer.MemoryCopy.
After digging a bit I've discovered the VK_MEMORY_PROPERTY_HOST_CACHED_BIT flag. I just tried to use it in the VkTexture constructor where a memory for a staging buffer is allocated (instead of VK_MEMORY_PROPERTY_HOST_COHERENT_BIT). And viola - the performance of copying is on par with Direct3D, and I can get comparable frame rate with both backends and didn't see any drawbacks.
// line 187 of VkTexture.cs
                _memoryBlock = _gd.MemoryManager.Allocate(
                    _gd.PhysicalDeviceMemProperties,
                    bufferMemReqs.memoryTypeBits,
                    VkMemoryPropertyFlags.HostVisible | VkMemoryPropertyFlags.HostCached,
                    true,
                    bufferMemReqs.size,
                    bufferMemReqs.alignment,
                    prefersDedicatedAllocation,
                    VkImage.Null,
                    _stagingBuffer);

It's the first ever time I deal with Vulkan thus I don't feel confident when it comes to working with it. Hence no pull request (yet).
Some questions go to more experienced Vulkan users:

Whether using "host cached" memory is a good idea for all staging resources? Probably, just for read by CPU?
Does it make sense to allow an explicit control over this flag for the Vulkan backend? On per-device, per-resource basis?
I've read that host cached memory may not be available on some devices. Does memory allocation with VK_MEMORY_PROPERTY_HOST_CACHED_BIT fails in that case or just ignores the flag? Is it implementation-specific?

Nice presentation describing various memory types in Vulkan and what to use them for
https://www.khronos.org/assets/uploads/developers/library/2018-vulkan-devday/03-Memory.pdf

I believe the code should just be updated to use a HOST_CACHED allocation if there exists a heap supporting it by the device, which is not guaranteed, and falling back to the current behavior otherwise.

Whether using "host cached" memory is a good idea for all staging resources? Probably, just for read by CPU?

It appears to be better for performance, but we still need to ensure that it is host coherent, as well. Without that, additional synchronization mechanisms are needed, which would make it a far more intrusive change. So the logic should check if HOST_CACHED | HOST_COHERENT is supported, and if it's not, fall back to HOST_COHERENT (current behavior). HOST_VISIBLE implied everywhere here.

Does it make sense to allow an explicit control over this flag for the Vulkan backend? On per-device, per-resource basis?

I don't think so. This is just an allocation optimization, the details of which are very hidden from the public API right now. If it's faster with no observable impact to the public interface, then we can just do it behind the scenes.

I've read that host cached memory may not be available on some devices. Does memory allocation with VK_MEMORY_PROPERTY_HOST_CACHED_BIT fails in that case or just ignores the flag? Is it implementation-specific?

The code will need to scan the memory types supported by the Vulkan device, with logic similar to VulkanUtil.FindMemoryType.

Thanks for your input @mellinoe
I'll get back to you with more questions if they arise or with a pull request otherwise.


Updated README.md, Added an Arrow instead of the ">"

Since this project is forked, I will be able to fetch and merge from your repository.
You won't need to setup a PR for that


Closes: #1

Updated validations.py python script.
Add "# -- coding: utf-8 --"
Add a list of special characters and amend the function validate_user(username, minlen)
In order to check if the first char of username is a special char.
Fixed the behavior of validate_user function in validations.py.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!

@googlebot I signed it!


Fix svd2swift incorrect extension name

Fixes a bug where using the '--namespace-under-device' svd2swift option would cause the tool to generate an extension for the type nested in the extension instead of the device being extended.
Fixes #87

@swift-ci test


Added a mini project

Changes made in this pull request

Added a mini project (Custom notepad).
Added my name to the contributors list.

@SVijayB good work.


#172 - City Scene

Closes #172
These changes break our city-drawing code into its own scene, and also break out the city label into its own scene.
Along the way I added a couple small optimizations, such as only redrawing the label background (which involves a lot of operations) if the label background has changed.
I also added in making it so only the capital has the capital star, and logging of turn times.

can this go in now or wait (ha) for Godot 4?

It can go in now, and may require some changes for Godot 4, particularly in the files in the C7/Map folder which are Godot Node2Ds.
I'll go ahead and merge it since Paul has approved it, with that note.


fix: Improve scroll velocity handling, add timer to reset scroll velocity

This PR adds timer to reset scroll velocity, when there was no scroll event for 500ms. It also prevents initial scroll to record velocity when using initial scroll index

Ah yes thanks for catching and fixing that! The scroll velocity stuff is still experimental and I guess I'd left it messy 😭


Add support for service groups

Steps to reproduce
I'm building an API with a lot of exposed services that belong to different groups.
For example, I have a group prospecting, with 8 different services inside it. What I want to achieve is URI structure like "prospects/persons/{id}", "prospects/companies/{id}" and so on.
To archive that, I'm defining my services like so:
  app.use('/prospects/persons', Object.assign(memory(), {docs}));

It works: API endpoints are available and SwaggerUI shows all endpoints.
But there are some serious problems.
Expected behaviour
What I want to do is to define docs for every of my sub-services and see them in Swagger. Also, ideally, I would like them to be displayed as different groups in SwaggerUI.
Actual behaviour
When I do so, they all appear under same tag "prospects" - this means that I can't set definitions for them (every definition is replacing the previous one) and can't group them by actual service name.
The problem is here because of automatic tags generation. It would be great to have an ability to:

set custom tags for service;
or at least support nested routes during generation.

System configuration
Module versions:

<EMAIL_ADDRESS><EMAIL_ADDRESS>

I've realised now that I can overcome definitions problem with using definitions field instead of definition. But there is still a problem with grouping by different tag.
P.S. I will be happy to fill a PR, but would like to discuss implementation before.
Maybe it makes sense to add a couple of recognised fields in docs schema?

tags (optional) - array of tags to assign to the service. If not given, current generation rules will apply.
definitionName (optional) - name to assign to definition. If not given, the first value from the array of tags will be used.


Let's register teaching languages for instructors

I was speaking to two of our french speaking instructors yesterday, and that got me wondering whether we ask people to register which language they would be comfortable teaching in. If we do, I though that could be used to select a first group of instructors to ask in case of workshops in countries where another language than English would be preferred.

Hello @karinlag.
We don't store information about languages that instructors feel comfortable teaching in.
@jduckles, @maneesha, @gvwilson do you feel we need that info too?
Thanks,
Piotr

Let me just add that we do, in fact, ask for workshop language in workshop request form.
We also provide a way for admins to find instructors from specific countries (I believe there's a correlation between used language and country person lives in :-) ).
If we start storing language info then we should:

store event language in events
store instructors language in their profile
ask for the language in profile update form

+1 - we've already been asked to get this for trainees so that they can
do their mentoring and mini-lesson in their preferred (human) language.

On 11/21/2015 12:12 PM, Piotr Banaszkiewicz wrote:

Let me just add that we do, in fact, ask for workshop language in
workshop request form.
We also provide a way for admins to find instructors from specific
countries (I believe there's a correlation between used language and
country person lives in :-) ).
If we start storing language info then we should:

Just to have it mentioned, the two french speaking instructors that
spurred this both live in Norway.
Karin

+1 I think it should allow for more than a single language, some instructors might be comfortable teaching in multiple languages. This is essentially the same as how we track the lessons someone is comfortable teaching it, we just want to track the languages they're comfortable teaching in as well. As Greg says, there may be utility in tracking the "preferred" language as well.

On Sat, Nov 21, 2015 at 07:15:19AM -0800, Jonah Duckles wrote:

+1 I think it should allow for more than a single language, some
instructors might be comfortable teaching in multiple
languages. This is essentially the same as how we track the lessons
someone is comfortable teaching it, we just want to track the
languages they're comfortable teaching in as well. As Greg says,
there may be utility in tracking the "preferred" language as well.

Here's how this works for HTML:

Language(s) of some content's intended audience with 1:
Content-Language = 1#language-tag
Like:
Content-Language: mi, en

Language(s) that can satisfy a given request 2:
Accept-Language = 1#( language-range [ weight ] )
Like:
Accept-Language: da, en-gb;q=0.8, en;q=0.7
would mean: "I prefer Danish, but will accept British English and
other types of English".

Glossing over the language range business 3, avoiding variants and
other fine-grained distinctions 4, and avoiding (for now) the
potentially useful script 5 and region 6, that would look
something like:
class Language(models.Model):
"""A language tag
  https://tools.ietf.org/html/rfc5646
  """
  name = models.CharField(max_length=STR_MED)
  language = models.CharField(max_length=STR_SHORT)

  def tag(self):
      return self.language

class LanguageQualification(models.Model):
"""What language can someone speak?
   https://tools.ietf.org/html/rfc7231#section-5.3.5
   """
   person = models.ForeignKey(Person)
   language = models.ForeignKey(Language)
   weight = models.FloatField(
       default=1,
       help_text='https://tools.ietf.org/html/rfc7231#section-5.3.1')

We could use student's LanguageQualification to match them with
workshops that speak their languages (“I see you prefer French, and we
have a workshop near you next year in French.  But we have a workshop
next month near you speaking Norwegian, and you know Norwegian with a
0.5 weight, so maybe you'd be interested in attending that instead?”)
and for matching instructors / helpers with workshops that need a
given language.

Almost all of the workshops I manage are in the US so I haven't really come across this.  It may come up across European and South American countries and bilingual Canada -- so I'd defer to the people managing those workshops as to how useful this would be and how it would work.
I am concerned though, with some political issues about how we identify languages.  @wking gave the example above about identifying British English -- I know in the US, if you specifically asked for someone speaking "American English" you could be facing a lawsuit.
I'm also not sure if we could easily get this information from instructors -- asking them to identify the languages they speak and weigh their ability.  We have a hard enough time getting instructors to update their profiles as is.  ("It'll be too hard" is not a reason not to do something, just something to consider in feasibility of implementation)

Just to have it mentioned, to me, English is english, I am able to
understand and be taught by americans, uk people, australians, and to my
mind there is no functional difference between the variants in a
teaching situation.
When it comes to instructors, I think we can settle for "which languages
would you be comfortable teaching in" with the default being english.
On 22/11/15 18:03, maneesha sane wrote:

Almost all of the workshops I manage are in the US so I haven't really
come across this. It may come up across European and South American
countries and bilingual Canada -- so I'd defer to the people managing
those workshops as to how useful this would be and how it would work.
I am concerned though, with some political issues about how we identify
languages. @wking https://github.com/wking gave the example above
about identifying British English -- I know in the US, if you
specifically asked for someone speaking "American English" you could be
facing a lawsuit.
I'm also not sure if we could easily get this information from
instructors -- asking them to identify the languages they speak and
weigh their ability. We have a hard enough time getting instructors to
update their profiles as is. ("It'll be too hard" is not a reason /not/
to do something, just something to consider in feasibility of
implementation)
—
Reply to this email directly or view it on GitHub
https://github.com/swcarpentry/amy/issues/582#issuecomment-158778448.

@wking do you want to tackle this one for v1.3 (dev period opens 2015-12-01, release && deployment early January)?

On Sun, Nov 22, 2015 at 09:03:14AM -0800, maneesha sane wrote:

I am concerned though, with some political issues about how we
identify languages.

I think punting to an RFC to which we add nothing (although we may
drop some things) puts us in a fairly safe political position.

I'm also not sure if we could easily get this information from
instructors -- asking them to identify the languages they speak and
weigh their ability…

So it will be even more difficult to get this information from
prospective students (except maybe trainee-instructors?).  I think
it's mostly a chicken-and-egg issue.  As long as most of our workshops
are in English or we just transport instructors within their language
region, it's not a big deal.  If we want to start pushing instructors
further afield, or run a French workshop on British Columbia (or an
English workshop in Quebec?), the more information we can get from
instructors, the less work we need from admins.  Since admin-time is
the only thing we buy, we get to pick whether it's easier to have
admins manages languages by hand, or have admins enter language
information into AMY for instructors, or have admins try to build an
instructor culture where folks manage their own AMY profiles.

On Sun, Nov 22, 2015 at 09:37:43AM -0800, karinlag wrote:

Just to have it mentioned, to me, English is english…

I got some funny faces asking for a peanut butter & jelly sandwich in
Australia ;).  But yeah, for programming, the differences within
English regions are probably not important.  I'm not sure about other
languages.  For example 1 has “many of which are not mutually
intelligible” for Chinese, but maybe all chinese-dialect speakers also
speak a more widely understood dialect.

When it comes to instructors, I think we can settle for "which
languages would you be comfortable teaching in" with the default
being english.

So I think we agree that we don't want regions, scripts, etc., for
now, because we don't see ourselves having the luxury of that level of
matching.  And maybe we're on the fence about weighting (although
“defaults to one” seems to handle our “will folks bother to tell us
this?” concern there).

On Sun, Nov 22, 2015 at 02:32:25PM -0800, Piotr Banaszkiewicz wrote:

@wking do you want to tackle this one for v1.3…

Sure :p.  It sounds like folks consider the Language and
LanguageQualification models I outlined in 1 either “looks good” or
“probably overkill, but as usable as a simpler model because of their
defaults”.  If folks want to explicitly green-light those models, or
propose changes to them, I should be able to get started on this.

@wking if you want to grab languages from somewhere, the list here looks decent: https://bitbucket.org/basaundi/django-languages/src/3b1bb169fd9bac289d499b0a6481482017357407/django_languages/languages.py?at=default&fileviewer=file-view-default (only languages with 2-letter code, the rest is overwhelming).

On Tue, Nov 24, 2015 at 09:14:36AM -0800, Piotr Banaszkiewicz wrote:

@wking if you want to grab languages from somewhere…

I'd rather just go to the source ;).  1 lists IANA as the registry
maintainer, 2 looks like that registry, and 3 describes the
entry-format in the registry.

… only languages with 2-letter code, the rest is overwhelming…

Just the 2-letter codes from 2 still gives us 190 languages, which
still seems overwhelming ;).  I expect the UI will have to be “just
enter your language code, or give us a string and we'll list
tags/descriptions for languages whose descriptions container your
string”.  Including 3-letter codes gives an additional 7903 entries
though, and “English” would create 23 hits:

en, English
aig, Antigua and Barbuda Creole English
ang, Old English (ca. 450-1100)
bah, Bahamas Creole English
bzj, Belize Kriol English
…

So I'm fine going with just the 2-letter codes for now, and adding
more obscure languages on an as-needed basis.

On 11/24/2015 6:14 PM, Piotr Banaszkiewicz wrote:

@wking https://github.com/wking if you want to grab languages from
somewhere, the list here looks decent:
https://bitbucket.org/basaundi/django-languages/src/3b1bb169fd9bac289d499b0a6481482017357407/django_languages/languages.py?at=default&fileviewer=file-view-default
(only languages with 2-letter code, the rest is overwhelming).

These codes fly a bit over my head :)
But I do get the impression that the admins would be able to say "hm,
here is a workshop that would like italian speaking instructors, lets
check if we have any and ask them", so I like.
karin

Unfortunately we missed the opportunity to do this before the update mail was sent out…

We'll be registering a new batch of instructors starting late
January/early February, so we can aim for then.


graph: Copy creates new edges

What are you trying to do?
Create a copy of a graph with graph.Copy.
What did you expect to happen?
My graph had custom nodes and edges values implementing encoding.Attributer.
I expected the copy to use the same values.
What actually happened?
My nodes were used to build the new graph. However the edges were created from scratch:
dst.SetEdge(dst.NewEdge(u, v))

Instead of something like the following:
dst.SetEdge(src.Edge(u.ID(), to.Node.ID()))

I'm not sure if this is a bug, sorry for the noise in advance!
What version of Go and Gonum are you using?

go: go version go1.11.4 linux/amd64
gonum: v0.0.0-20190123113241-dd4cc715c58a (go.mod)

Does this issue reproduce with the current master?
Yes.

@kortschak, thank you so much!

Sorry it took so long; it required thought about the best way to tackle it. I think we have that.


deployment failing on workspace asclab-la-y3wfs4anr2gxe

I keep getting error
{
"status": "Failed",
"error": {
"code": "Conflict",
"target": "name",
"message": "The workspace name 'asclab-la-y3wfs4anr2gxe' is not unique"
}
}
I have gone thru my entire subscription and I can't find this resource even after I deleted the RGs and started over.
Looking for guidance on how to get around this error
thx

more info

here is screen shot of error

more info

here is screen shot of error

more info

I am going to delete both RGs and then see if I can find a way to force delete the workspace prior to 14 days.

more info

I am going to delete both RGs and then see if I can find a way to force delete the workspace prior to 14 days.

found hard delete options - https://docs.microsoft.com/en-us/azure/azure-monitor/platform/delete-workspace

found hard delete options - https://docs.microsoft.com/en-us/azure/azure-monitor/platform/delete-workspace

Hi @schnegja
has the documentation you've found solved your issue?

Hi @schnegja
has the documentation you've found solved your issue?

yes, I had to recreate the RG, then recover the LA workspace into the RG, then do a force delete.  Then I waited a few mins and relaunched the deploy from module one.  Deployment just finished no errors.

yes, I had to recreate the RG, then recover the LA workspace into the RG, then do a force delete.  Then I waited a few mins and relaunched the deploy from module one.  Deployment just finished no errors.

Sounds good. Closing this issue, then.

Sounds good. Closing this issue, then.


Extend task module size directions

Document the changes made in PR 160

:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/concepts/task-modules/task-modules-overview.md
:warning:Warning

Details

msteams-platform/concepts/task-modules/task-modules-overview.md

Line 17: [Warning] Invalid file link:(~/concepts/task-modules/task-modules-tabs).
Line 18: [Warning] Invalid file link:(~/concepts/cards/cards).
Line 18: [Warning] Invalid file link:(~/concepts/task-modules/task-modules-bots).
Line 29: [Warning] Invalid file link:(~/resources/schema/manifest-schema#icons).
Line 30: [Warning] Invalid file link:(~/resources/schema/manifest-schema#name).
Line 32: [Warning] Invalid file link:(~/concepts/task-modules/task-modules-tabs#example-submitting-the-result-of-a-task-module).
Line 44: [Warning] Invalid file link:(~/concepts/task-modules/task-modules-bots?branch=task-modules-csharp-examples#invoking-a-task-module-via-taskfetch).
Line 44: [Warning] Invalid file link:(~/concepts/task-modules/task-modules-bots#the-flexibility-of-task-submit).
Line 44: [Warning] Invalid file link:(~/concepts/task-modules/task-modules-bots#the-flexibility-of-task-submit).
Line 56: [Warning] Invalid file link:(~/resources/schema/manifest-schema#validdomains).
Line 205: [Warning] Invalid file link:(~/resources/schema/manifest-schema#id).
Line 205: [Warning] Invalid file link:(~/resources/schema/manifest-schema#validdomains).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[bde] Updated to 3.117.0

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download
[x] The "supports" clause reflects platforms that may be fixed by this new version
[x] Any fixed CI baseline entries are removed from that file.
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

Fixes https://github.com/microsoft/vcpkg/issues/18937
Replaces https://github.com/microsoft/vcpkg/pull/31644
Changes since the prior PR:

Remove the cxx17 feature flag. This is an incomplete solution to the issue but avoids allowing conflicting feature flags in libraries.
Disable OSX build. Linux is the primary target - I'd like to fix others in future version of this port. The existing BDE version in vcpkg is ancient & seemingly non-functional.

CC'ing @BillyONeal and @FrankXie05 from the previous PR

Thank you so much! Sorry for the nitpicks...

Moving the PR to draft. Please set it to "ready for review" once you respond.

It looks like the CI might take ten hours, I'll mark this ready for review with the idea that my local builds work with these changes.

/azp run

We are fixing the failure of dcmtk:x64-windows and dcmtk:x64-windows-static-md.

Waiting for https://github.com/microsoft/vcpkg/pull/32049 to be merged.

Moving the PR to draft. I will set it to "ready for review" once the CI issue fixed.

Also should provide the usage file in port bde

/azp run

Is this something known?

Yes. The fix was just merged: #32257.

/azp run

REGRESSION: halide:x86-windows failed with BUILD_FAILED. If expected, add halide:x86-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.

REGRESSION: halide:x86-windows failed with BUILD_FAILED. If expected, add halide:x86-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.

Baseline issue. Cf. https://github.com/microsoft/vcpkg/pull/32259#issuecomment-1610830059

The referenced issue is resolved, hopefully this passes now too?

@adamncasey Please merge into master branch. :)

Unfortunately, it looks like bde made changes to these so it isn't straightforward to devendor them: https://github.com/bloomberg/bde/commit/5e057e052c28fa9c7c575c06f835e454879bad37 "with Bloomberg extensions"
In particular bde is requiring that a pkgconfig file exists for ryu and it does not upstream. We could add that but without an understanding of what the other changes are I'm not comfortable messing with this on my own.
@adamncasey @cppguru Unfortunately I believe this forces us to remove bde from the curated registry as it can't be installed with other supported ports, and won't be link compatible with anything using the real ryu or pcre2.

I filed https://github.com/bloomberg/bde/issues/295

Oleg and Bloomberg is working on this.  I am not, and I'm on leave at the moment. Could you please make an issue in the bde repo or email the team.


Link to "Add React to an Existing App" corrected.

Link to "Add React to an Existing App" corrected. It was pointing to "/docs/add-react-to-a-new-app.html".
Thank you for the PR! Contributors like you keep React awesome!
Please see the Contribution Guide for guidelines:
https://github.com/reactjs/reactjs.org/blob/master/CONTRIBUTING.md
If your PR references an existing issue, please add the issue number below:
#464

Oops, thanks.


reset supply not setting divisible/non-divisible correctly

seems that with the most recent version of freewallet, the "reset supply" is always choosing divisible instead of non-divisible regardless of what the user enters.

fixed in 0.9.34 release


main return exit_code to caller

When execute exit -1 the main function must to return exit_code to first bash called minishell
./minishell << EOF
exit 1
EOF
echo $? // must be 1
./minishell << EOF
exit a
EOF
echo $? // must be 2
./minishell << EOF
exit -1
EOF
echo $? // must be 255

Waiting review @carlosrocha-dev
PR #52


Inconsistency for Bing Web Search API with Java SDK docs

Description
I have an Azure subscription with Bing Search ressource (free tier):

I have written a small Spring service to interact with that ressource.
I followed these documentations steps:

https://learn.microsoft.com/en-us/azure/cognitive-services/bing-web-search/ (Then click Use Java in Bing Web Search)
https://learn.microsoft.com/en-us/azure/cognitive-services/bing-web-search/quickstarts/client-libraries?pivots=programming-language-java (Deprecated, so I clicked on the new proposed link, bellow)
https://learn.microsoft.com/en-us/bing/search-apis/ (Then click References for Bing Web Search API)
https://learn.microsoft.com/en-us/bing/search-apis/bing-web-search/reference/endpoints (Then click in the menu Quickstarts, SDKs, Java)

So I followed this tutorial to use the Java SDK with Bing Web Search API: https://learn.microsoft.com/en-us/bing/search-apis/bing-web-search/quickstarts/sdk/web-search-client-library-java.
The SDK return an authentication error (HTTP 401).
Because the proposed SDK uses the URL https://api.cognitive.microsoft.com instead of https://api.bing.microsoft.com/. But the references (https://learn.microsoft.com/en-us/bing/search-apis/bing-web-search/reference/endpoints) still using https://api.bing.microsoft.com/.
But
Steps to reproduce
My small service:
@Service
public class BingSearchService {

    private BingWebSearchAPI client;

    public BingSearchService(@Value("${cloud.azure.bing.apikey}") String apiKey) {
        client = BingWebSearchManager.authenticate(apiKey);
    }

    public String search(final String searchQuery) throws Exception {
        SearchResponse webData = client.bingWebs().search()
                .withQuery(searchQuery)
                .withCount(50)
                .execute();

        String results = webData.webPages().value().toString();
        return results;
    }
}

As a result, I get an error:
{"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource."}}

In the source code of the SDK, I found that it calls the URL https://api.cognitive.microsoft.com/bing/v7.0/.
So, for an additional test, I tried different URLs to verify my subscription key:
❯ curl "https://api.cognitive.microsoft.com/bing/v7.0/images/search?q=microsoft" -H "Ocp-Apim-Subscription-Key: [REDACTED]"
{"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource."}}

And:
❯ curl "https://api.bing.microsoft.com/v7.0/images/search?q=microsoft" -H "Ocp-Apim-Subscription-Key: [REDACTED]"
# The results are showing

So, it confirms that the subscription key is working, but only for api.bing.microsoft.com.
Suggestion
I found this repository with the working implementation and URL:

https://github.com/microsoft/bing-search-sdk-for-java:

https://github.com/microsoft/bing-search-sdk-for-java/blob/main/samples/sdk/WebSearchSample/README.md
https://github.com/microsoft/bing-search-sdk-for-java/tree/main/samples

Maybe the actual documentation should be refreshed if Bing Search ressource is used intead of Cognitive Service (or similar) ?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 73a384e5-1447-14f0-e55d-37d7c4c6ea1f
Version Independent ID: 2fe02dc2-b1db-14c1-543f-add375503d40
Content: Quickstart: Use a Bing Web Search client library - Azure Cognitive Services
Content Source: articles/cognitive-services/Bing-Web-Search/quickstarts/client-libraries.md
Service: cognitive-services
Sub-service: bing-web-search
GitHub Login: @aahill
Microsoft Alias: aahi

@cosad3s
Thanks for your feedback! We will investigate and update as appropriate.

@cosad3s
Thanks for your contribution. Please add your ideas in below link, so our production team can review it and update the same.
Ideas · Community (azure.com)

@cosad3s
We are going to close this thread. if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


[Enhancement] multi left join do not push down

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
Version: doris-2.1.5-rc02-a04583c9a1
Step 1: create three tables with same schema, and insert some example data:
CREATE TABLE `a` (
  `k1` INT NOT NULL,
  `k2` INT NOT NULL,
  `v` INT NULL
) ENGINE=OLAP
DUPLICATE KEY(`k1`, `k2`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`k1`) BUCKETS 1
PROPERTIES ("replication_num"="1");

CREATE TABLE `b` (
  `k1` INT NOT NULL,
  `k2` INT NOT NULL,
  `v` INT NULL
) ENGINE=OLAP
DUPLICATE KEY(`k1`, `k2`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`k1`) BUCKETS 1
PROPERTIES ("replication_num"="1");

CREATE TABLE `c` (
  `k1` INT NOT NULL,
  `k2` INT NOT NULL,
  `v` INT NULL
) ENGINE=OLAP
DUPLICATE KEY(`k1`, `k2`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`k1`) BUCKETS 1
PROPERTIES ("replication_num"="1");

INSERT INTO a VALUES(1,2,3),(4,5,6);
INSERT INTO b VALUES(1,2,3),(4,5,6);
INSERT INTO c VALUES(1,2,3),(4,5,6);

Step 2: Use one left join like bellow, we found y.k1=1  generated and pushed down to table b
> explain shape plan select * from a as x left join b as y on x.k1 = y.k1 where y.k1=1;
+--------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                              |
+--------------------------------------------------------------------------------------------------------------+
| PhysicalResultSink                                                                                |
| --PhysicalDistribute[DistributionSpecGather]                                                                 |
| ----hashJoin[INNER_JOIN] hashCondition=((x.k1 = y.k1)) otherCondition=() build RFs:RF0 k1->[k1];RF1 k1->[k1] |
| ------filter((x.k1 = 1))                                                                                |
| --------PhysicalOlapScan[a] apply RFs: RF0 RF1                                                               |
| ------PhysicalDistribute[DistributionSpecHash]                                                               |
| --------filter((y.k1 = 1))                                                                                |
| ----------PhysicalOlapScan[b]                                                                                |
+--------------------------------------------------------------------------------------------------------------+

Problem 1:
But if we replace table y with another left join, we found y.k1=1 only pushed to join node, not pushed to table b and c
> explain shape plan select * from a as x left join (select c.k1 as k1 from b left join c on b.k1=c.k1) as y on x.k1 = y.k1 where x.k1=1;
+-------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                     |
+-------------------------------------------------------------------------------------+
| PhysicalResultSink                                                                  |
| --PhysicalDistribute[DistributionSpecGather]                                        |
| ----hashJoin[LEFT_OUTER_JOIN] hashCondition=((x.k1 = y.k1)) otherCondition=()       |
| ------filter((x.k1 = 1))                                                            |
| --------PhysicalOlapScan[a]                                                         |
| ------PhysicalDistribute[DistributionSpecHash]                                      |
| --------PhysicalProject                                                             |
| ----------filter((y.k1 = 1))                                                        |
| ------------hashJoin[LEFT_OUTER_JOIN] hashCondition=((b.k1 = k1)) otherCondition=() |
| --------------PhysicalProject                                                       |
| ----------------PhysicalOlapScan[b]                                                 |
| --------------PhysicalDistribute[DistributionSpecHash]                              |
| ----------------PhysicalProject                                                     |
| ------------------PhysicalOlapScan[c]                                               |
+-------------------------------------------------------------------------------------+

Problem 2:
If left table is come from UNION, the predict do not push down to table c
> explain shape plan with x as(select * from a union select * from b) select * from x left join c on x.k1=c.k1 where x.k1=1;
+-------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                               |
+-------------------------------------------------------------------------------+
| PhysicalResultSink                                                            |
| --PhysicalDistribute[DistributionSpecGather]                                  |
| ----hashJoin[LEFT_OUTER_JOIN] hashCondition=((x.k1 = c.k1)) otherCondition=() |
| ------hashAgg[GLOBAL]                                                         |
| --------PhysicalDistribute[DistributionSpecHash]                              |
| ----------hashAgg[LOCAL]                                                      |
| ------------PhysicalUnion                                                     |
| --------------PhysicalDistribute[DistributionSpecExecutionAny]                |
| ----------------PhysicalProject                                               |
| ------------------filter((a.k1 = 1))                                          |
| --------------------PhysicalOlapScan[a]                                       |
| --------------PhysicalDistribute[DistributionSpecExecutionAny]                |
| ----------------PhysicalProject                                               |
| ------------------filter((b.k1 = 1))                                          |
| --------------------PhysicalOlapScan[b]                                       |
| ------PhysicalDistribute[DistributionSpecReplicated]                          |
| --------PhysicalOlapScan[c]                                                   |
+-------------------------------------------------------------------------------+

Solution
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

https://github.com/apache/doris/pull/39450/


Unfuck events maybe i hope

About the PR
This PR blindly changes a bunch of the event weights in hopes that it does something.
Do I know what these numbers do? No
Do I have any way to test it without wasting my time? No
Why / Balance
The game has very little emergent gameplay right now, at least compared to SS13. Stuff like space dragon and zombie outbreak appears practically never. Also there's no way to test these properly without merging so like. You know.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog
:cl:

tweak: Changed a bunch of events to be more common.

tempted to say hold off on merging this until i work out metrics for events so we can see how this PR changes them

i lied. you can merge this whenever


Running on node?

I am having some trouble figuring out how to get this to work on node. I notice in your readme there are mentions of this throughout, but it seems like running on node needs some extra steps other than just do npm/yarn install. Could you write out some more detail instructions specifically for node?
It looks like I need to cd into the janus-gateway-js folder in node and run gulp? Then configure a third party webrtc adaptor? Is that correct for node use?

Installing this library Node ist supported. Just run:
npm i janus-gateway-js
On node, there will be no UI or similar. The lib then just provides an easy way to communicate to the Janus API.
https://www.npmjs.com/package/janus-gateway-js


Support for Database Level Integrity Features

Perhaps related to #671.
Is there support/openness to utilizing more of the capabilities of the underlying database? #690 would allow for some more of this manually, but it would be nice to have the generators apply these, especially for #183.
My motivation is that getting integrity constraints right is "$:*ing tough, and the kind folks behind Postgres, Maria, MYSQL, etc have put enormous effort into making it correct and optimize performance.

FOREIGN KEY CONSTRAINTS
NOT NULL
CHECK
CASCADES (Useful for Many 2 Many)
UNIQUE
etc.

ex.)
 schema "comments" do
    belongs_to :post, Post, null: false, delete: :cascade
    timestamps
  end

SQL
"CREATE TABLE IF NOT EXISTS comments(
      post_id INTEGER NOT NULL REFERENCES posts(id) ON DELETE CASCADE,
      inserted_at TIMESTAMP NOT NULL,
      updated_at TIMESTAMP NOT NULL)"

Yes, definitely.
We already support unique and references in migration (foreign key constraints), no?
I am ok with supporting cascade and check depending on how well it is supported across databases. Can you please do a survey and investigate the syntax for those for Sqlite3, MySQL, PostgreSQL and MSSQL?
Also, those shouldn't be in the schema, but in the migration.

@josevalim They do work, I just didn't find them :embarrassed: I'll create a new issue with results of syntax and summarizing scope for check/cascade. The following is for posterity.
REFERENCES
(@zmoshansky RTFM closer), references work just great! Turns out the phoenix generator does not use them though. ex.) phoenix.gen.model Cat cat owner:belongs_to name:string
In Migration Do:
add :owner_id, references(:owners)

UNIQUE
For future readers, UNIQUE CONSTRAINT doesn't exist ATM, however an INDEX that is UNIQUE across multiple columns can be created, and they are only very subtly different. Differences.
In Migration Do:
    create index(:cat, [:name, :owner_id], unique: true)

@zmoshansky / @josevalim i care a lot about database level integrity, and would like to open an issue on creating unique constraint (and potentially contributing to it) on the migration level, how do i go about that? 🙏🏼 Thanks!

Hi @wutania! 👋  unique coinstraints and similar are already supported in the ecto_sql project (note this issue is 8 years old)


Change updateElement to updateElements

This is a very minor performance improvement. It removes bunch of function calls. It also makes it easier to do some setup just once like getting the scales instead of doing once per element, which means we don't have to worry so much about trying to cache variables

I'm not quite sure this is worth the gained performance. Did you get any measurable results?
Should minify a bit better though, can you check that too?
It would be really nice to have some really basic automatic comparisons between PR and master. Like size and some performance checks.

Size on master: 173878
Size with PR: 173773

The performance is extremely variable. I get +/-50% between runs. Unless it's a huge change I wouldn't be able to pick it up
There is some improvement to minification:
Size on master: 173878
Size with PR: 173773

Needs rebasing, other than that looks good to me.

Build failed, I think you need to rebase to more recent commit (it is 4 commits behind still)

I rebased again, but it's still failing on the new clipping tests. I can't figure out what's going on

That:
https://github.com/chartjs/Chart.js/blob/64b47762429ac5f5752d5a1126baf10f3282369f/src/core/core.datasetController.js#L725
Using me._xScale and me._yScale that are not set anymore. That probably fails for other controllers too, so should really get the scales there.

Thanks @kurkle ! That did the trick
Btw, it doesn't look to me like _xScale is set for any type other than line (this PR doesn't change that)

hm. with using a single updateElements function is makes it more difficult to extend the updateElements method in sub classes. e.g. In various plugins I override the updateElement method to add some customized styling.

@sgratzl can you point me to the repo where you think this would be most challenging? Would it be the chartjs-chart-geo that you linked to?

@sgratzl can you point me to the repo where you think this would be most challenging? Would it be the chartjs-chart-geo that you linked to?

it is possible, just wanna raise the awareness that API changes (such adding new parameters or this change) can effect plugin developers

Yes, we're currently rethinking the API to improve performance and make other improvements. We'll be providing a migration guide: https://github.com/chartjs/Chart.js/blob/master/docs/getting-started/v3-migration.md


[Android] Backport some upstream commits to fix our device tests.

There have been some changes in M41 that causes our device tests to fail on the ICS devices we have. This pull request backports some fixes I made upstream for them so that we get back to a usable state.
We still upload some binaries (like device_forwarder) to the device more often than we actually need (see this Chromium bug I reported for more information), but it is nothing too huge.

@darktears

RSBM


Moving errors into pkg/errors

Is your feature request related to a problem? Please describe.
We put errors package at the root of the repository for simplicity.
However, with some recent discussion about eventually use the errors package in templates of scaffolded app, I think we should now move it into pkg/errors to be more future proof and avoid having errors in chain importing the errors package
pkg would represent in the future a list of utility packages that can be used to build Cosmos-SDK blockchain

Do you mean a new pkg package in this repo, or the existing ignite/cli/pkg package ?

Personally as a user I don't expect to see utilities packages inside a repo called modules.

True but it's more like modules with a build-in environment for the development of those modules and to test them. Which includes reusable packages like errors, the primary usage of these packages are in the modules of the repo. ignite/pkg would be for more universal package, not for those where the primary usage would be for the modules.

OK for errors indeed, but I had also in mind cosmoscmd and this one is not used in modules.


How to hide sensitive environment variables values in Portainer UI?

Currently, all secretes are showing up for each container on Portainer. Is there any way to hide them or show *** (mask value) for specific fields?

Sorry, need more information. Where are the secrets shown please. Screenshots would help here.

Please see the picture. I am talking about the environment variables for docker containers (microservices). The DB username and password are showing up as plain text. Wondering if there is a way hide/mask some of the environment variables.

We are using Portainer 1.24.0 docker image.

Oh, that is why Docker provide “secrets”, as a way to hold sensitive information. You need to stop using environment variables for sensitive information, and instead switch to using swarm secrets to accomplish the same.
Environment variables are always stored in plain text and we wont ever obfuscate them.

@ncresswell I have additional question - how secrets are encrypted (and where) if Kubernetes Secrets (portainer-cs v2.x+) are used inside Portainer?
Thank you for explanation.

@ncresswell I have additional question - how secrets are encrypted (and where) if Kubernetes Secrets (portainer-cs v2.x+) are used inside Portainer?
Thank you for explanation.


s/WebhookChannel2023/WebHookChannel2023/

I'm currently drafting a draft PR for @jaxoncreed's https://github.com/o-development/solid-notification-client
I noticed that we use different capitalizations for WebSocket and Webhook. I think this might lead to some mistakes, one example is in CSS which uses WebHook instead of Webhook
https://github.com/CommunitySolidServer/CommunitySolidServer/blob/main/src/server/notifications/WebHookChannel2023/WebhookChannel2023Type.ts#L17
I think we should quickly consider changing capitalization in the spec, vocab, and context to have matching

WebSocket
WebHook

This way CSS could just keep it as currently implemented and we will lower chances of similar misspelling happening in the future.

No, this is not a mistake. That is how the two standards "brand" themselves. WebSocket and Webhook!
Our job is to reflect what protocol is being used as a notification channel.

Makes sense @CxRes
Once we have a test suite available it should prevent any typos in implementations that pass it.


TextInput on:input misbehaviour

Hi,
I was trying to change from:
<input type="text" bind:value={user.email} on:input={updateUser} >
to
<TextInput bind:value={user.email} on:input={updateUser} />
First is absolutely functional. But second is behaving like so: every 2 letters typed is disappearing...
As it was linked to firebase I'm testing with, I tried a more simple example like below:
<script>
...
let textInputData = "";

const alertText = () => {
  console.log(textInputData);
}
...
</script>
...
<TextInput bind:value={textInputData} on:input={alertText} placeholder="Type anything" />
<input type="text" bind:value={textInputData} on:input={alertText} placeholder="Type anything">
...

Now I don't see letter disappearing but console log with TextInput is always one letter behind while log always match with html input.
Do you know this issue? Any workaround?

Is it possible to do something like this?
$: if (textInputData) console.log('change', textInputData);

@metonym Many thanks. Just works ;-)


Merge in hharte's Northstar changes

This adds hharte's fix to the hard-sector support in the firmware, and the Northstar encoder/decoder and NSI image reader and writer.

Hello David,
Thank you, the changes look pretty good.  I found a couple issue with the .textpb configurations, and fixed them in #267.  With this change, reading and writing works for DS/DD (350K), SS/DD (175K) and SS/SD (87.5K) disks.
Take care,
Howard


[stable/concourse] optionally pass additional cli arguments to workers

What this PR does / why we need it:

Allow configuring command-line arguments that will be passed to the Concourse workers
Arguments are given as a single string and will be passed verbatim (no escaping)

Possible use-case:
Some users are running in a corporate environment with private certificate authorities. Using the --certs-dir option trust relationships can be established to those CA and, thus, having to disable tls-validation can be avoided (which not all resources might even support).

CLA should be signed now

@AndreasBurger I'd like to stick with env vars which can be set via .Values.worker.env:
worker:
  env:
    - name: CONCOURSE_CERTS_DIR
      value: /some-dir

You can see what env vars are supported with docker run -it concourse/concourse [web|worker] --help; all cmd line args have corresponding env vars. In the future I'd like to clean up the chart so that only env vars are used.

/close


Add valueOrEmpty'

λ> valueOrZero False (Just 'a')
<interactive>:4:1:
    No instance for (Monoid Char) arising from a use of ‘valueOrZero’

λ> valueOrEmpty False (Just 'a')
*** Exception: user error (mzero)

λ> valueOrEmpty' False (Just 'a')
Nothing

Does this belong in p, or is there something out there for it already?
@jystic @charleso @olorin

Not that it's in the right spot, but: http://hackage.haskell.org/package/attoparsec-<IP_ADDRESS>/docs/Data-Attoparsec-Combinator.html#v:option

No opinion on this either way :fencesit:

No objection to this. :+1:

Closing, not useful enough. I shoehorned my code into valueOrEmpty and I've long forgotten the one other time I wanted this function.


uquihash.eu - server not work

my 30 workers not works.

Duplicate of #80

No. Not work with -l equihash.eu.nicehash.com:3357. Write this:

Now works. To fix?

Our servers were down at maintenance if this issue still persists reopen it.


Evaluation procedure ended abruptly.

Hi. I have finished training on the REVIDE dataset and I tried to evaluate the iter_40000.pth checkpoint. However, after the warning:"[Shape mismatch] lq: torch.Size([1, 54, 3, 900, 1354]), gt: torch.Size([1, 54, 3, 1800, 2708])", the evaluation procedure ended abruptly. How to solve this problem? Thank you.

Make sure you have enough GPU memory to run the inference.

Thanks for your reply.


Release?

Any chance for a release soon? I've got this addon-guard notification, and noticed the ember-get-config dependency version is updated to 0.3.0 on the master branch.
ember-get-config
----------------------------------------
my-app
├─┬ ember-cli-mirage
│ └──<EMAIL_ADDRESS>(cacheKey: b37f276e40fd793ce66813dfb160f912, runtime: true)
├─┬ ember-cli-moment-shim
│ └──<EMAIL_ADDRESS>(cacheKey: ffb52ed19a11426f4028bc2838e5cf87, runtime: true)
└─┬ ember-power-calendar-moment
  └─┬ ember-cli-moment-shim
    └──<EMAIL_ADDRESS>(cacheKey: ffb52ed19a11426f4028bc2838e5cf87, runtime: true)

There's been some issues with the dev build that has been blocking a release: https://github.com/miragejs/ember-cli-mirage/pull/2081#issuecomment-735860430
If you are motivated + have time to help out, that would probably be the fastest way to get to the release! I'm personally focused on other things at the moment so can't help out on this.

These above mentioned issues have been taken care of

the docs not working on ember-cli-mirage for master branch)
acceptance tests not working with ember try because addon docs only supports 3+

I think we are clear to make a release.  While I dont recall anything that was specifically a breaking change, I believe Sam wanted to bump the major version number if I recall

https://github.com/miragejs/ember-cli-mirage/pull/2025 Think we reverted this so we need to make sure we hvaen't changed anything since. That's what the major was for.

Looks like we dropped node 6 and 8 in our package.json so we do need to bump.
I'll defer to @Turbo87 here.
https://github.com/miragejs/ember-cli-mirage/compare/v1.1.8...master#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519L98

yep, seems reasonable :)

Ok yep I'd say go ahead and release and just bump to 2.0 and note why!

if you're talking to me, I don't have npm permissions 😅

Haha meant @cah-briangantzler because he's been kinda driving this from something he needed, but happy to add you to npm if you'd like (with no expectation of doing any maintenance work!)

This is you correct? https://www.npmjs.com/~turbo87

Figured we haven't gotten to the point of people impersonating npm users but never hurts to ask 😁

yeah, as far as I know that should be me :D

gotcha :) Added!

v2.0.0 is running through TravisCI now and will hopefully be available on npm soon.
turns out I probably didn't need the npm access if CI is doing the release for me 😅

Oh yeah - derr. That's one of the reasons I did that, so I didn't have to add people to npm 🙈 Oh well!

alright, v2.0.0 is up on npm now. closing this issue :)

Thanks for doing this. Was a little nervous to do it, first time.
I updated my app to 2.0.0 no problem.
Can not yet confirm upgrading to qunit 5 as https://github.com/san650/ember-cli-page-object needs an update as well


Correction for raw affiliation Université Paris IV Sorbonne, Gripic,

Correction needed for raw affiliation Université Paris IV Sorbonne, Gripic,
raw_affiliation_name: Université Paris IV Sorbonne, Gripic,
new_rors: ;04p8das24
previous_rors:
works_examples: W2519612744
contact: 2dc79d221dd2b0dbcd6040e9e8ae3eb9:539fe73f3905a0c270d443d910 @ sorbonne-universite.fr

This issue was accepted and ingested by the OpenAlex team on 2024-11-01. The new affiliations should be visible within the next 7 days.


Fix breakage for Linux

Partially revert commit https://github.com/StevenBlack/hosts/commit/0a99194b2844bbc026ab62da69798fdc34815a79 to fix #2015

Failed checks are because of ./updateReadme.py:65:121: E501 line too long (130 > 120 characters) which is not part of my change.

Thank you for chasing this down @DPTJKKVH.  Awesome!  Merging!
This fixes #2015.


Add support for sparse arrays inside dask arrays

[x] Closes #1113
[ ] Tests added
[ ] Release note added (or unnecessary)

I got a little excited. Maybe this will be 0.10.0.

This is a good POC for dask arrays holding cupy and cupyx arrays
This definitely could be more performant. We could do better handling of writing at the very least. I.e. don't build a SparseDataset and append, instead do something like:

hold indptr + a chunk sized buffer for indices and data in memory
for each chunk generated, write storage chunks until we can't write an entire chunk.
Fill memory buffer with remaining data.
Generate next matrix chunk start by filling memory buffer, and writing to storage.

TODO:

[x] Indexing tests
[x] Concatenation tests
[ ] Views?
[x] Work around or skip transpose

https://github.com/scipy/scipy/issues/19161

I've now tried making it faster using the approach above. It's faster, but like 20% for default zarr settings. Here's that implementation:

def write_dask_sparse(f, k, elem, _writer=None, dataset_kwargs=MappingProxyType({})):
    if k in f:  # TODO: Delete in actual implementaion, handled by machinery
        del f[k]  # TODO: Delete in actual implementaion, handled by machinery

    def chunk_slice(start: int, stop: int) -> tuple[slice | None, slice | None]:
        result = [slice(None), slice(None)]
        result[axis] = slice(start, stop)
        return tuple(result)

    def resize_and_write(storage: ZarrArray | H5Array, buffer: np.ndarray):
        prev_shape = storage.shape[0]
        storage.resize((prev_shape + buffer.shape[0],))
        storage[prev_shape:] = buffer

    if isinstance(f, H5Group) and "maxshape" not in dataset_kwargs:
        dataset_kwargs = dict(maxshape=(None,), **dataset_kwargs)

    chunktype = elem._meta # 0 dim instance of spmarix with correct dtype
    sparse_format = chunktype.format

    # Determine axis to concatenate along from sparse format
    if sparse_format == "csr":
        axis = 0
    elif sparse_format == "csc":
        axis = 1
    else:
        raise NotImplementedError(
            f"Cannot write dask sparse arrays with format {sparse_format}"
        )
    axis_chunks: tuple[int, ...] = elem.chunks[axis]
    chunk_start, chunk_stop = 0, 0

    # Start initializing output
    group = f.create_group(k)
    group.attrs["shape"] = elem.shape

    group.attrs["encoding-type"] = f"{sparse_format}_matrix"  # TODO: Delete in actual implementaion, handled by machinery
    group.attrs["encoding-version"] = "0.1.0"  # TODO: Delete in actual implementaion, handled by machinery

    # Initialize final storage arrays + intermediate buffers
    data_storage = group.create_dataset("data", shape=(0,), dtype=chunktype.data.dtype, **dataset_kwargs)
    storage_chunksize = data_storage.chunks[0]  # This should probably actually be determined by shape
    data_buffer = np.zeros(storage_chunksize, dtype=chunktype.data.dtype)
    indices_storage = group.create_dataset("indices", shape=(0,), dtype=chunktype.indices.dtype, **dataset_kwargs)
    indices_buffer = np.empty(storage_chunksize, dtype=chunktype.indices.dtype)

    indptr = np.zeros(elem.shape[axis] + 1, dtype=np.int64)
    buffer_offset = 0

    for chunk_size in axis_chunks:
        chunk_start = chunk_stop
        chunk_stop += chunk_size

        matrix_chunk = elem[chunk_slice(chunk_start, chunk_stop)].compute()

        local_offset = 0  # offset into current chunks indices/ data array
        nnz = matrix_chunk.nnz

        indptr[chunk_start + 1: chunk_stop + 1] = indptr[chunk_start] + matrix_chunk.indptr[1:]

        # Fill remaining buffer
        if buffer_offset != 0:
            buffer_remaining = storage_chunksize - buffer_offset
            local_offset = min(buffer_remaining, nnz)

            new_buffer_offset = buffer_offset + local_offset

            indices_buffer[buffer_offset : new_buffer_offset] = matrix_chunk.indices[:local_offset]
            data_buffer[buffer_offset : new_buffer_offset] = matrix_chunk.data[:local_offset]

            if new_buffer_offset == storage_chunksize:
                resize_and_write(indices_storage, indices_buffer)
                resize_and_write(data_storage, data_buffer)
                buffer_offset = 0
            else:
                buffer_offset = new_buffer_offset

        # Write any full sized chunks
        n_full_chunks, n_remaining = divmod(nnz - local_offset, storage_chunksize)
        if n_full_chunks > 0:
            local_slice = slice(local_offset, local_offset + n_full_chunks * storage_chunksize)
            resize_and_write(indices_storage, matrix_chunk.indices[local_slice])
            resize_and_write(data_storage, matrix_chunk.data[local_slice])
            local_offset += n_full_chunks * storage_chunksize

        # Write rest to buffer
        if n_remaining > 0:
            indices_buffer[:n_remaining] = matrix_chunk.indices[local_offset : local_offset + n_remaining]
            data_buffer[:n_remaining] = matrix_chunk.data[local_offset : local_offset + n_remaining]
            buffer_offset = n_remaining

    # Write whatever is left in the buffer
    if buffer_offset:
        indices_storage.append(indices_buffer[:buffer_offset])
        data_storage.append(data_buffer[:buffer_offset])

    group.create_dataset("indptr", data=indptr, **dataset_kwargs)

Some timing results:

 Setup
X = sparse.random(
    100_000,
    10_000,
    format="csr",
    density=0.01,
    random_state=np.random.default_rng(),
)
X_dask = da.from_array(X, chunks=(1000, 5000))

z = zarr.open("test.zarr", "w")

%timeit write_elem(z, "from_memory", X, dataset_kwargs={"chunks": 10_000})
%timeit write_elem(z, "original_implementation", X_dask, dataset_kwargs={"chunks": 10_000})
%timeit write_dask_sparse(z, "new_implementation", X_dask, dataset_kwargs={"chunks": 10_000})

In memory:  1.43 s ± 60.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Naive Dask: 2.36 s ± 88.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
New impl:   2.08 s ± 88.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

From profiling, it seems like a lot of the difference is from:

Every time the array is resized, the metadata is updated
Time for dask compute

I think a lot of this could be made up with more write buffering.
But also this is way more complex.
Transpose
Tranpose is an issue since scipy doesn't let you pass axes to transpose and dask always passes that argument.
I've opened a PR to scipy to fix, but we'll need to work around this until then. Either with a fix or disallowing this.
I think the fix should look like:
def transpose_by_block(dask_array: da.Array) -> da.Array:
    b = dask_array.blocks
    b_raveled = b.ravel()
    block_layout = np.zeros(b.shape, dtype=object)

    for i in range(block_layout.size):
        block_layout.flat[i] = b_raveled[i].map_blocks(lambda x: x.T, chunks=x.shape[::-1])

    return da.block(block_layout.T.tolist())

Transpose each block of the dask array
Transpose the block structure of the dask array

Indexing
dask.Array.vindex is broken when the meta array is a sparse matrix. Also maybe when it's a dense np.matrix class, but that is less important.

@selmanozleyen pinging you for a look at this. It's pretty close to done I think, just 3 failing tests left. WDYT?


Evaluate the rollout mechanism of nested mutations

Evaluate and come up with a way so that the new DAB versions support nested mutations and also remain backward compatible

Evaluate and come up with an approach for the development workflow/lifecycle for the feature.

Evaluate whether a feature flag is needed for nested mutations after release - i.e provide users the ability to hide/turn off the nested mutations feature if needed?

We will be merging all the sub-tasks (Schema Generation, Authorization, Request validation, Database Query Generation) into the feature branch: dev/NestedMutations.
We would need a feature flag for enabling nested mutations (by default support for nested mutations would be off).

We will be merging all the sub-tasks (Schema Generation, Authorization, Request validation, Database Query Generation) into the feature branch: dev/NestedMutations.
We would need a feature flag for enabling nested mutations (by default support for nested mutations would be off).

Shouldn't the feature be enabled by default?

all questions answered here. Feature flag work continued in separate issue and other two questions decided.


Update tailwind.css

Pull Request Checklist

[x] Description: Briefly describe the changes in this pull request.
[x] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.

[x] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?

Description
This pull request combines the font-family declaration for html and pre elements to save space and avoid repetition.
Changed
Combined the font-family declaration for html and pre elements to save space and avoid repetition.

Documentation: Not applicable in this case.

Dependencies: Not applicable in this case.

LGTM, Thanks!


Avoid sleep(100) when shutting down (and measuring startup)

The Startup_empty_startup benchmark which executes simple hello world seems to be influenced by Thread.sleep(100) in Serialization manager skewing the result by 3-5% as the last 101ms the application is doing nothing:

Avoid Thread.sleep and replace it by proper synchronization primitives. Consider rewriting the SerializationManager to Java.

I'll handle this as part of https://github.com/enso-org/enso/pull/8924/commits/300972db59f31a33994aeac9f54dfec7785f418e while introducing the SerializationPool.


How Anti-afk works?

How Anti-afk works?

I think it makes use of the command +right which means that you cant be kicked by the server for being afk as it doesnt classify you as afk . However you can still be kicked if your teammates vote to kick you.

It sends different UserCmd::buttons every other tick, so the server thinks we aren't afk.


When downloading, include the statistics.

When downloading file, it would be great to have the statistics such us:

number of views,
number of likes and dislikes,
number of comments

One of the usage would be to order the downloaded videos by revelancy.
It's possible to implement it?

This is already implemented quite some time ago.


Spanner connector

Ability to support GSP Spanner as database

Connection tests passed on top of private GCP trial

Both connection & sql generation tests (postgres on top of spanner) passed

Integration tests passed

Integration tests passed localy

Integration tests passed localy

Good to merge. Most of the code has been modularized to fit the new pattern. Integration tests for connection and sql gen pass locally. NOTE: Tc run is pending.

Integration tests passed:

All integration tests are green

Good to merge! Please ask David to merge.