Enhancement: Add shared library support for AIX

I am aware the AIX is not an officially supported platform - but the core change needed to support it is quite small - and updated DSOFLAGS and supporting LDFLAGS.
Would be great to have the basic support for AIX back in play.
While I, personally, prefer having shared library support in (fat) archives - having the SYSV support of \.so files is a great step forward.
Will submit a PR asap.

FYI: The PR #390 has been tested on AIX 5.3 TL7 (for build) and AIX 7.1 (used in OSS project requiring CUPS support). Compiler is xlC-v11.

@aixtools If you are willing to take on the responsibility of periodically testing and submitting PRs to keep CUPS building on AIX, I can move AIX from the "unsupported" list to the "supported by their developers" (middle tier) list. The only reason why AIX support was dropped was because the last time I had an AIX system was 15 years ago and there were no other developers/resources available to continue it.

Hi,
If you need an extra tester for AIX, you can contact me.
Currently, there is an issue with the CUPS on AIX but I will open a new issue for it.
Stef


Extend cron job to also check and build torchserve-kfs docker images

Thank you for doing this! I also use torchserve with kserve and require python>=3.10, making the official torchserve-kfs image a hassle to use (though a lot easier now that I found your images).
Here you can find the Dockerfile:
https://github.com/pytorch/serve/tree/master/kubernetes/kserve
These get built and pushed to: https://hub.docker.com/r/pytorch/torchserve-kfs/tags
Thanks!

Hi @cjidboon94
thank you for your feedback, happy to hear it's useful for you.
I'll take a look those other builds and see what I can do - thank you for the suggestion!

Hi @cjidboon94
I just took a look at the Dockerfile and the /kubernetes/kserve/build_image.sh script.
I think I can pull a relatively easy patch to extend the current cron job.
The only problem is that the mentioned build_image.sh script has the values of BASE_IMAGE hardcoded and in general does not look in such a great shape like to rely on it at the moment (for example, the script also has harcoded the instruction to push on docker build, which is cumbersome).Plus, the docker github workflows have been broken for a while on the torchserve repo. So it is diffucult to know what their plans are.
Taking all that into account I don't feel confident to put  time and effort on the serve repo to fix from the ground up the build_image.sh script because I don't know in which direction they want to go with that.
Back to your problem, looking at the kserve/Dockerfile, I think I would just use an image from https://hub.docker.com/r/fabridamicelli/torchserve/tags as base image and add the little pieces (like 10 lines of code ) from the /kubernetes/kserve/Dockerfile that I need to run the specific application you're running (by the way, I just tried to run that docker build and also didn't work because of some grpc stuff – once again, I think that part of the project is not yet in great shape).

Hi @fabridamicelli !
Much appreciated for your swift and detailed answer. I indeed managed to get a solution by replacing the BASE_IMAGE values in the build_image.sh script to replace with one of your images which is what your proposed solution also looks like. The issue with the grpc is that you need to initialize the submodule (`git submodule init && git submodule update``) that has the required grpc dep. Then it should be able to build

Hi @cjidboon94
Ok, I am glad to hear you found a workaround and thank you for the tip.
If it's fine for you I will now close this issue since you have a working solution and for the time being no further work is planned  on this one.
Feel free to open a new issue in the future when the torchserve kserve repo is more stable and I will be happy to extend the workflow if that's what we need.
Best,
F


Few issues with OCP 4.11

Hello,
I tried to deploy this demo on a 4.11 OCP cluster.
Pre-requisites are installed :
$ pip3 list | grep -e kubernetes -e openshift -e jmespath
jmespath            1.0.1
kubernetes          24.2.0
openshift           0.13.1

$ ansible --version
ansible [core 2.13.4]
  config file = None
  configured module search path = ['/Users/slallema/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/6.4.0/libexec/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/slallema/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.10.7 (main, Sep 14 2022, 22:38:23) [Clang 14.0.0 (clang-14<IP_ADDRESS>)]
  jinja version = 3.1.2
  libyaml = True

I have a first issue with the install.sh phase and the ocp4-post-acs task :
TASK [ocp4-post-acs : Get the secret that contains the token of sa pipeline] ***************************************************************
ok: [localhost] => {"ansible_facts": {"token_sa_pipeline_secret": []}, "changed": false}

TASK [ocp4-post-acs : Get token in the secret for the sa pipeline and decode] **************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: list object has no element 0\n\nThe error appears to be in '/Users/slallema/GIT/github.com/slallemand/devsecops-demo/bootstrap/roles/ocp4-post-acs/tasks/post_ci.yaml': line 68, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Get token in the secret for the sa pipeline and decode\n  ^ here\n"}

PLAY RECAP *********************************************************************************************************************************
localhost                  : ok=70   changed=6    unreachable=0    failed=1    skipped=3    rescued=0    ignored=0

Anyway, i did try to start the pipeline with the ./demo.sh start but the pipeline is failing at the build-image task.
I have those errors :
STEP-GEN-ENV-FILE

2022/10/05 13:03:28 warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
UID uid=1001(1001) gid=0(root) groups=0(root),1000660000
Generated Env file
------------------------------
MAVEN_CLEAR_REPO=false
MAVEN_MIRROR_URL=http://nexus:8081/repository/maven-public/
------------------------------
STEP-GENERATE

2022/10/05 13:03:29 warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
UID uid=1001(1001) gid=0(root) groups=0(root),1000660000 s2i build spring-petclinic/target image-registry.openshift-image-registry.svc:5000/openshift/java:11 --image-scripts-url image:///usr/local/s2i --as-dockerfile /gen-source/Dockerfile.gen --environment-file /env-params/env-file
STEP-BUILD

Error: error writing "0 0<PHONE_NUMBER>\n" to /proc/29/uid_map: write /proc/29/uid_map: operation not permitted
level=error msg="error writing \"0 0<PHONE_NUMBER>\\n\" to /proc/29/uid_map: write /proc/29/uid_map: operation not permitted"
level=error msg="(unable to determine exit status)"
STEP-PUSH-TAG

2022/10/05 13:03:31 Skipping step because a previous step failed
STEP-PUSH-LATEST

2022/10/05 13:03:32 Skipping step because a previous step failed

Could that be relative to the install error ?
Any idea on that ?

Quick update : I tried on a 4.9 cluster without any issue so really shoulb be relative to 4.11

hi @slallemand, 4.11 of OpenShift uses 1.24 version of k8s where several changes where introduced, such as the deprecation of PSP, and several major changes that affects also to this demo. Works like a charm in versions below 4.11. We'll fix asap this repo in order to make the demo work in all the versions.
Thanks for open the issue!

@rcarrata Just tested on OCP 4.11 cluster provisioned by RHDP (Red Hat Demo Platform) and the install script fails at:
TASK [ocp4-post-acs : Get token in the secret for the sa pipeline and decode] *************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: list object has no element 0\n\nThe error appears to be in '/Users/username/test-devsecops/devsecops-demo/bootstrap/roles/ocp4-post-acs/tasks/post_ci.yaml': line 68, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Get token in the secret for the sa pipeline and decode\n  ^ here\n"}

Any idea ? There is no token_sa_pipeline_secret in the cicd namespace but only pipeline-token-xxxx

Seeing the same issue on ROSA. Can you help pls ?

Everything works on the current version of ARO...
Client Version: 4.12.1 Kustomize Version: v4.5.7 Server Version: 4.10.54 Kubernetes Version: v1.23.12+8a6bfe4

Any plans to fix this on OCP 4.11+?

fixed and tested with the latest 4.13. Grabbing the pipeline-token directly instead of the pipeline sa.
Closing this issue.


Concatenation of builtin functions(sincos(x)) is valid.

The following expression sincos(1) is seen as valid for exp4j
Although the evaluation of the expression is done correct, meaning, even if the expression is written in the following manner sincos(1) the code evaluates it as sin(cos(1)), shouldn't the shunting yard algorithm detect that sincos(x) function is not written in a valid form and when calling the validate() method return false?

Hey @alexinul, Thanks for submitting this issue. This is a bug probably due to the implicit multiplication feature.


More helpfull error messages.

Sometimes program stopped and didn't know why. Mostly Lower limit reached. But would it be possible to take some text to error message? F.e. "upper/lower limit reched", For Fibonacci, "Stopped cause lvl. xx was reached" and so on.
That would be realy helpful

These aren't error messages, they're notifications. It's been added to the todo list


buffer.getChannelData(0) returns null on  Chromium: 77.0.3865.90 (Brave)

Sorry if duplicate - couldn't find anything on this.
I noticed when trying to create a PingPongDelay, played around in the browser console until I got down to the buffer.getChannelData call in Context.js.
It looks like the buffer is initializing correctly, and has a channel, but getChannelData not working.
Tested on Firefox and it works fine there.
see logs when this fiddle is run:
https://jsfiddle.net/m201gh4j/
(broken for both tone/next and tone/latest)
Thanks for all the hard work you've put into this - def one of the coolest libraries out there :)

Also this makes no sense to me but it might help: the issue doesn't appear when using parceljs to serve files locally. It does still appear when using parcel build or just linking the library directly in an html file.

Thanks for the issue. When i run the code i don't see any errors in the console. can you point me to the steps you took to reproduce the error that you got?
Here's a little demo of loading a buffer and printing getChannelData(0) to the console.

The fiddle works for me in firefox, prints null in Brave (Chromium 77). I downloaded Chrome and it works there – looks like it's an issue with Brave/their version of Chromium. Probably not worth your time!

interesting. i'm seeing that too on Brave. My guess is it's a content security policy. In this case since the buffer is loaded from another server, it will not let you inspect the bytes, but the buffer is still usable in that it'll let you play the audio, just not look at the data.
I just tested the same code out running on a local server with Brave and i was able to download the buffer and getChannelData. So that support my theory about the heightened security sandboxing which Brave adds.
The solution is to make sure that you're downloading audio from the same origin as your server. Let me know if you're able to validate my theory by testing audio files downloaded from the same origin.

I wasn't loading a buffer from an audio file though– I'm pretty sure it was creating one in memory as part of calling new Tone.PingPongDelay (all on the same frontend).
Not sure if I'm misunderstanding you though?

maybe i misunderstood as well. How are you generating the buffer exactly?
If you could provide a code example of how you reproduced the error, that'd be helpful. then we'd know we're talking about the same thing. :)

yeah so the fiddle I linked at the top does have the error for me - I was trying to boil it down to what call was actually failing.
My code that was originally producing the error was:
const ppDelay = new PingPongDelay(0.5, 0.5).toMaster()
but it invokes buffer.getChannelData(0) somewhere in the creation of the delay object, which returns null which is what causes it to throw
Does that make sense? sorry I should have started at the highest level.

ok i see now where that's coming from. yeah still seems like this is a content security policy because of Brave's heightened javascript security. Are you getting this same error when you're running your code locally on Brave? because it works fine for me as long as all the code originates from the same origin.
For example this code runs fine on brave which is using a PingPongDelay but where all of source code originates from the same source.

with device recognition shields up: works in localhost, doesn't work when just opening local file, doesn't work in production (pretty sure all coming from same origin but netlify might be doing something funky)
with those shields down: works everywhere
I'm not too worried about supporting brave with those specific shields up (I think they break threejs as well), so you can feel free to close this
thanks for the help :)

i'm glad that we have this documented here, but if a user has a security/privacy policy set on their browser which blocks some Web Audio features, i don't think there is much that we can do to support Tone.js in their environment.
Going to close this issue. If there are workarounds or suggestions, or some documentation on which features do and don't work, it'd be great to link to those somewhere.


Bad output height calculation for empty output

Execute github issue query that returns some results
Change the query so that it will return 0 results
Rerun it
The output goes away but the same amount of space is consumed

@roblourens are you seeing this issue 100% of the time? I can't reproduce it.

Yeah. I can debug it

Envelop.kubernates(service.threat.circle.bash)
Moot(critical.devops.circle)
Uptick(refUser.data.circle)
Down(refUset.data.circle)
Blogs (enterprise.servo.namely)
Ad(offers.inside.metadata)
Bash(repositorys.tier5.bat)

It's gone


Missing DataStyle

Here is the attribute DataStyle missing,
https://doc.stride3d.net/latest/en/manual/scripts/serialization.html
DataStyle for ScalarStyle and DataStyle explanation is missing

Wrong repo? I can move it to stride-docs 🙂

oh ye ups


Smart detection rule name rename?

The smart decection rule with name 'extension_resourceutilizationextensionspackage' is not recognized as a valid rulename anymore. Was this rule renamed?
OPERATION ID  CA032AA53004A87A
TRACKING ID  cbff1489-ee77-446b-85a5-9d9ffc77f1ef
STATUS  BadRequest
STATUS MESSAGE  {
"code": "BadRequest",
"message": "There is no such rule with the given name: "extension_resourceutilizationextensionspackage"",
"innererror": {
"diagnosticcontext": "fc722c71-ec68-4a15-a995-1c799c8e5c1f",
"time": "2019-06-27T07:15:21.6417479Z"
}
}
PROVISIONING STATE  Failed
TIMESTAMP  6/27/2019, 9:15:21 AM
DURATION  1 second
TYPE  Microsoft.Insights/components/ProactiveDetectionConfigs

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4b22256b-6d3e-f8df-9309-477e181f6a8f
Version Independent ID: b54c4e18-60e2-470b-5e12-ff25a4ecf143
Content: Configure Azure Application Insights smart detection rule settings with Azure Resource Manager templates
Content Source: articles/azure-monitor/app/proactive-arm-config.md
Service: application-insights
GitHub Login: @harelbr
Microsoft Alias: harelbr

Hi Kris,
This Smart Detection rule, which was in a 'preview' state, has been deprecated.
Apologies for the inconvenience.
Thanks,
Harel

Thanks fo the info.
Is it possible to update the docs please.

Yes, the docs are currently in the process of being updated.

@harelbr Thanks for sharing the info. I will go ahead and close the issue for now.


⚠ Bofrost.fr has degraded performance

In f0d3c59, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1042 ms

Resolved: Bofrost.fr performance has improved in 49c7d47 after 10 minutes.


Accept of list of monitor ids in findings and alerts request dtos (#277)

Backports from main to 2.x
Signed-off-by: Surya Sashank Nistala<EMAIL_ADDRESS>

Codecov Report

Merging #278 (311d7f6) into 2.x (082c776) will increase coverage by 0.09%.
The diff coverage is 92.30%.

@@             Coverage Diff              @@
##                2.x     #278      +/-   ##
============================================
+ Coverage     72.37%   72.47%   +0.09%
- Complexity      662      666       +4
============================================
  Files           106      106
  Lines          4406     4417      +11
  Branches        601      601
============================================
+ Hits           3189     3201      +12
+ Misses          966      965       -1
  Partials        251      251

Impacted Files
Coverage Δ

...arch/commons/alerting/action/GetFindingsRequest.kt
92.85% <85.71%> (+5.90%)
:arrow_up:

...search/commons/alerting/action/GetAlertsRequest.kt
96.66% <100.00%> (+0.83%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Question about Complex datatype support

The inductor backend of Pytorch2.0 does not officially support Complex data types yet (https://github.com/pytorch/pytorch/issues/93424), just wondering if hidet has the same limitation currently, or not?
If it does does it rely on other parts of Pytorch2.0 (e.g., the inductor, dynamo etc.) to fully support complex, or can complex support be added separately?

Hi @wcqc,
Yes, we do not officially support complex data types yet.
Hidet does not reply on any pytorch components to work (e.g., you can install hidet and use its other frontends like onnx).
Can I know what networks are you using? It would be great if you could share a self-contained script to run a complex-value network and I am happy to add some basic support for that in hidet.

Hi @yaoyaoding ,
Thank you for the swift reply!
torch.compile() failed for me when I used the inductor backend of PyTorch2.0 due the missing support of complex dtype in the inductor and many other places in PyTorch2.0. The above was more of a general question on this with regards to hidet.
On the other hand more specifically, I'm trying to accelerate code based on this library: https://github.com/mit-han-lab/torchquantum, which mainly involves matrix multiplications where the matrices contain complex numbers). An example program can be found here:
https://github.com/mit-han-lab/torchquantum/blob/main/examples/simple_mnist/mnist_example.py, and when I run this with torch.compile(model, backend='hidet'), the error I'm getting is:
/venvs/pt2py3.10/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 675, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: hidet_backend raised TypeError: avg_pool2d() missing 2 required positional arguments: 'stride' and 'padding', occurred when calling avg_pool2d with
    args: (<hidet.Tensor object at 0x7ff987e95840>, 6)
  kwargs: {}
avg_pool2d is defined at
  File "/venvs/pt2py3.10/lib/python3.10/site-packages/hidet/graph/frontend/torch/register_functions.py", line 175

Set torch._dynamo.config.verbose=True for more information

You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True

Now my question is: Does it look like hidet would work with a lib such as TorchQuantum above and provide acceleration (after fixing the above error)? Or as it currently stands hidet would not work with TorchQuantum due to the lack of complex dtype support?
Thanks again.

Hi @wcqc,
Sorry, I still can not reproduce the error you encountered as I do not have the backgroud of quantum machine learning/simulation and do not know how to put the torch.compile in the example.
Again, a self-contained example that can be directly run would be helpful.
In general, if the torch dynamo could extract the computation graph for us, we are happy to add the support. I expect it would not take long to make it functionally work, as long as torch dynamo can extract the graph for us and no strange operator occurred. But to achieve the good performance, we still need to add some complex-value number related schedule template (as a complex64 or complex128 number have 8 and 16 bytes which is larger than the heavily optimized data type like float32 and float16 (4 bytes and 2 bytes).

This is a script to reproduce the above error, it can be run with: python script.py, it still requires the torchquantum library.
Is this sufficient?
`import hidet
import torch
import torch.nn.functional as F
import torch.optim as optim
import argparse
import random
import numpy as np
import torchquantum as tq
from torchquantum.plugins import (
tq2qiskit_measurement,
qiskit_assemble_circs,
op_history2qiskit,
op_history2qiskit_expand_params,
)
from torchquantum.datasets import MNIST
from torch.optim.lr_scheduler import CosineAnnealingLR
class QFCModel(tq.QuantumModule):
class QLayer(tq.QuantumModule):
def init(self):
super().init()
self.n_wires = 4
self.random_layer = tq.RandomLayer(
n_ops=50, wires=list(range(self.n_wires))
)
        # gates with trainable parameters
        self.rx0 = tq.RX(has_params=True, trainable=True)
        self.ry0 = tq.RY(has_params=True, trainable=True)
        self.rz0 = tq.RZ(has_params=True, trainable=True)
        self.crx0 = tq.CRX(has_params=True, trainable=True)

    def forward(self, qdev: tq.QuantumDevice):
        self.random_layer(qdev)

        # some trainable gates (instantiated ahead of time)
        self.rx0(qdev, wires=0)
        self.ry0(qdev, wires=1)
        self.rz0(qdev, wires=3)
        self.crx0(qdev, wires=[0, 2])

        # add some more non-parameterized gates (add on-the-fly)
        qdev.h(wires=3)  # type: ignore
        qdev.sx(wires=2)  # type: ignore
        qdev.cnot(wires=[3, 0])  # type: ignore
        qdev.rx(
            wires=1,
            params=torch.tensor([0.1]),
            static=self.static_mode,
            parent_graph=self.graph,
        )  # type: ignore

def __init__(self):
    super().__init__()
    self.n_wires = 4
    self.encoder = tq.GeneralEncoder(tq.encoder_op_list_name_dict["4x4_u3rx"])

    self.q_layer = self.QLayer()
    self.measure = tq.MeasureAll(tq.PauliZ)

def forward(self, x, use_qiskit=False):
    qdev = tq.QuantumDevice(
        n_wires=self.n_wires, bsz=x.shape[0], device=x.device, record_op=True
    )

    bsz = x.shape[0]
    x = F.avg_pool2d(x, 6).view(bsz, 16)
    devi = x.device

    if use_qiskit:
        # use qiskit to process the circuit
        # create the qiskit circuit for encoder
        self.encoder(qdev, x)
        op_history_parameterized = qdev.op_history
        qdev.reset_op_history()
        encoder_circs = op_history2qiskit_expand_params(self.n_wires, op_history_parameterized, bsz=bsz)

        # create the qiskit circuit for trainable quantum layers
        self.q_layer(qdev)
        op_history_fixed = qdev.op_history
        qdev.reset_op_history()
        q_layer_circ = op_history2qiskit(self.n_wires, op_history_fixed)

        # create the qiskit circuit for measurement
        measurement_circ = tq2qiskit_measurement(qdev, self.measure)

        # assemble the encoder, trainable quantum layers, and measurement circuits
        assembled_circs = qiskit_assemble_circs(
            encoder_circs, q_layer_circ, measurement_circ
        )

        # call the qiskit processor to process the circuit
        x0 = self.qiskit_processor.process_ready_circs(qdev, assembled_circs).to(  # type: ignore
            devi
        )
        x = x0

    else:
        # use torchquantum to process the circuit
        self.encoder(qdev, x)
        qdev.reset_op_history()
        self.q_layer(qdev)
        x = self.measure(qdev)

    x = x.reshape(bsz, 2, 2).sum(-1).squeeze()
    x = F.log_softmax(x, dim=1)

    return x

def train(dataflow, model, device, optimizer):
for feed_dict in dataflow["train"]:
inputs = feed_dict["image"].to(device)
targets = feed_dict["digit"].to(device)
    outputs = model(inputs)
    loss = F.nll_loss(outputs, targets)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print(f"loss: {loss.item()}", end="\r")

def valid_test(dataflow, split, model, device, qiskit=False):
target_all = []
output_all = []
with torch.no_grad():
for feed_dict in dataflow[split]:
inputs = feed_dict["image"].to(device)
targets = feed_dict["digit"].to(device)
        outputs = model(inputs, use_qiskit=qiskit)

        target_all.append(targets)
        output_all.append(outputs)
    target_all = torch.cat(target_all, dim=0)
    output_all = torch.cat(output_all, dim=0)

_, indices = output_all.topk(1, dim=1)
masks = indices.eq(target_all.view(-1, 1).expand_as(indices))
size = target_all.shape[0]
corrects = masks.sum().item()
accuracy = corrects / size
loss = F.nll_loss(output_all, target_all).item()

print(f"{split} set accuracy: {accuracy}")
print(f"{split} set loss: {loss}")

def main():
parser = argparse.ArgumentParser()
parser.add_argument(
"--static", action="store_true", help="compute with " "static mode"
)
parser.add_argument("--pdb", action="store_true", help="debug with pdb")
parser.add_argument(
"--wires-per-block", type=int, default=2, help="wires per block int static mode"
)
parser.add_argument(
"--epochs", type=int, default=2, help="number of training epochs"
)
args = parser.parse_args()

if args.pdb:
    import pdb

    pdb.set_trace()

seed = 0
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)

dataset = MNIST(
    root="./mnist_data",
    train_valid_split_ratio=[0.9, 0.1],
    digits_of_interest=[3, 6],
    n_test_samples=75,
)
dataflow = dict()

for split in dataset:
    sampler = torch.utils.data.RandomSampler(dataset[split])
    dataflow[split] = torch.utils.data.DataLoader(
        dataset[split],
        batch_size=256,
        sampler=sampler,
        num_workers=8,
        pin_memory=True,
    )

use_cuda = torch.cuda.is_available()
device = torch.device("cuda" if use_cuda else "cpu")

_model = QFCModel().to(device)
model = torch.compile(_model, backend='hidet')

n_epochs = args.epochs
optimizer = optim.Adam(model.parameters(), lr=5e-3, weight_decay=1e-4)
scheduler = CosineAnnealingLR(optimizer, T_max=n_epochs)

if args.static:
    # optionally to switch to the static mode, which can bring speedup
    # on training
    model.q_layer.static_on(wires_per_block=args.wires_per_block)

for epoch in range(1, n_epochs + 1):
    # train
    print(f"Epoch {epoch}:")
    train(dataflow, model, device, optimizer)
    print(optimizer.param_groups[0]["lr"])

    # valid
    valid_test(dataflow, "valid", model, device)
    scheduler.step()

# test
valid_test(dataflow, "test", model, device, qiskit=False)

# run on Qiskit simulator and real Quantum Computers
try:
    from qiskit import IBMQ
    from torchquantum.plugins import QiskitProcessor

    # firstly perform simulate
    print(f"\nTest with Qiskit Simulator")
    processor_simulation = QiskitProcessor(use_real_qc=False)
    model.set_qiskit_processor(processor_simulation)
    valid_test(dataflow, "test", model, device, qiskit=True)

    # then try to run on REAL QC
    backend_name = "ibmq_lima"
    print(f"\nTest on Real Quantum Computer {backend_name}")
    # Please specify your own hub group and project if you have the
    # IBMQ premium plan to access more machines.
    processor_real_qc = QiskitProcessor(
        use_real_qc=True,
        backend_name=backend_name,
        hub="ibm-q",
        group="open",
        project="main",
    )
    model.set_qiskit_processor(processor_real_qc)
    valid_test(dataflow, "test", model, device, qiskit=True)
except ImportError:
    print(
        "Please install qiskit, create an IBM Q Experience Account and "
        "save the account token according to the instruction at "
        "'https://github.com/Qiskit/qiskit-ibmq-provider', "
        "then try again."
    )

if name == "main":
main()

Thanks for the script!
I will have a look when I have time and try to add the missing operator when needed.

Hi @wcqc,
I have added the missing operators in #271. But there is a bug that I can not fix from hidet side. You can run the script and get the subsequent error message. It is likely a bug of torch dynamo (I have checked the outputs of hidet backend for the received sub-graph, everything looks good). I did observe that there are a lot of fusion opportunity for this network (e.g., see the outs/graphs/graph_3, we can fuse the whole sub-graph into a single kernel).
You can also try using hidet's onnx frontend if you can export the inference task as an onnx model.
import hidet
import torch
import torch.nn.functional as F
import torch.optim as optim
import argparse
import random
import numpy as np

import torchquantum as tq
from torchquantum.plugins import (
    tq2qiskit_measurement,
    qiskit_assemble_circs,
    op_history2qiskit,
    op_history2qiskit_expand_params,
)

from torchquantum.datasets import MNIST
from torch.optim.lr_scheduler import CosineAnnealingLR

hidet.option.cache_dir('./outs/cache')
hidet.torch.dynamo_config.dump_graph_ir('./outs/graphs')
hidet.torch.dynamo_config.print_input_graph()

class QFCModel(tq.QuantumModule):
    class QLayer(tq.QuantumModule):
        def __init__(self):
            super().__init__()
            self.n_wires = 4
            self.random_layer = tq.RandomLayer(
                n_ops=50, wires=list(range(self.n_wires))
            )

            # gates with trainable parameters
            self.rx0 = tq.RX(has_params=True, trainable=True)
            self.ry0 = tq.RY(has_params=True, trainable=True)
            self.rz0 = tq.RZ(has_params=True, trainable=True)
            self.crx0 = tq.CRX(has_params=True, trainable=True)

        def forward(self, qdev: tq.QuantumDevice):
            self.random_layer(qdev)

            # some trainable gates (instantiated ahead of time)
            self.rx0(qdev, wires=0)
            self.ry0(qdev, wires=1)
            self.rz0(qdev, wires=3)
            self.crx0(qdev, wires=[0, 2])

            # add some more non-parameterized gates (add on-the-fly)
            qdev.h(wires=3)  # type: ignore
            qdev.sx(wires=2)  # type: ignore
            qdev.cnot(wires=[3, 0])  # type: ignore
            qdev.rx(
                wires=1,
                params=torch.tensor([0.1]),
                static=self.static_mode,
                parent_graph=self.graph,
            )  # type: ignore

    def __init__(self):
        super().__init__()
        self.n_wires = 4
        self.encoder = tq.GeneralEncoder(tq.encoder_op_list_name_dict["4x4_u3rx"])

        self.q_layer = self.QLayer()
        self.measure = tq.MeasureAll(tq.PauliZ)

    def forward(self, x, use_qiskit=False):
        qdev = tq.QuantumDevice(
            n_wires=self.n_wires, bsz=x.shape[0], device=x.device, record_op=True
        )

        bsz = x.shape[0]
        x = F.avg_pool2d(x, 6).view(bsz, 16)
        devi = x.device

        if use_qiskit:
            # use qiskit to process the circuit
            # create the qiskit circuit for encoder
            self.encoder(qdev, x)
            op_history_parameterized = qdev.op_history
            qdev.reset_op_history()
            encoder_circs = op_history2qiskit_expand_params(self.n_wires, op_history_parameterized, bsz=bsz)

            # create the qiskit circuit for trainable quantum layers
            self.q_layer(qdev)
            op_history_fixed = qdev.op_history
            qdev.reset_op_history()
            q_layer_circ = op_history2qiskit(self.n_wires, op_history_fixed)

            # create the qiskit circuit for measurement
            measurement_circ = tq2qiskit_measurement(qdev, self.measure)

            # assemble the encoder, trainable quantum layers, and measurement circuits
            assembled_circs = qiskit_assemble_circs(
                encoder_circs, q_layer_circ, measurement_circ
            )

            # call the qiskit processor to process the circuit
            x0 = self.qiskit_processor.process_ready_circs(qdev, assembled_circs).to(  # type: ignore
                devi
            )
            x = x0

        else:
            # use torchquantum to process the circuit
            self.encoder(qdev, x)
            qdev.reset_op_history()
            self.q_layer(qdev)
            x = self.measure(qdev)

        x = x.reshape(bsz, 2, 2).sum(-1).squeeze()
        x = F.log_softmax(x, dim=1)

        return x

def train(dataflow, model, device, optimizer):
    for feed_dict in dataflow["train"]:
        inputs = feed_dict["image"].to(device)
        targets = feed_dict["digit"].to(device)

        outputs = model(inputs)
        loss = F.nll_loss(outputs, targets)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        print(f"loss: {loss.item()}", end="\r")

def valid_test(dataflow, split, model, device, qiskit=False):
    target_all = []
    output_all = []
    with torch.no_grad():
        for feed_dict in dataflow[split]:
            inputs = feed_dict["image"].to(device)
            targets = feed_dict["digit"].to(device)

            outputs = model(inputs, use_qiskit=qiskit)

            target_all.append(targets)
            output_all.append(outputs)
        target_all = torch.cat(target_all, dim=0)
        output_all = torch.cat(output_all, dim=0)

    _, indices = output_all.topk(1, dim=1)
    masks = indices.eq(target_all.view(-1, 1).expand_as(indices))
    size = target_all.shape[0]
    corrects = masks.sum().item()
    accuracy = corrects / size
    loss = F.nll_loss(output_all, target_all).item()

    print(f"{split} set accuracy: {accuracy}")
    print(f"{split} set loss: {loss}")

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--static", action="store_true", help="compute with " "static mode"
    )
    parser.add_argument("--pdb", action="store_true", help="debug with pdb")
    parser.add_argument(
        "--wires-per-block", type=int, default=2, help="wires per block int static mode"
    )
    parser.add_argument(
        "--epochs", type=int, default=2, help="number of training epochs"
    )

    args = parser.parse_args()

    if args.pdb:
        import pdb

        pdb.set_trace()

    seed = 0
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)

    dataset = MNIST(
        root="./mnist_data",
        train_valid_split_ratio=[0.9, 0.1],
        digits_of_interest=[3, 6],
        n_test_samples=75,
    )
    dataflow = dict()

    for split in dataset:
        sampler = torch.utils.data.RandomSampler(dataset[split])
        dataflow[split] = torch.utils.data.DataLoader(
            dataset[split],
            batch_size=256,
            sampler=sampler,
            num_workers=8,
            pin_memory=True,
        )

    use_cuda = torch.cuda.is_available()
    device = torch.device("cuda" if use_cuda else "cpu")

    _model = QFCModel().to(device)
    model = torch.compile(_model, backend='hidet')

    n_epochs = args.epochs
    optimizer = optim.Adam(model.parameters(), lr=5e-3, weight_decay=1e-4)
    scheduler = CosineAnnealingLR(optimizer, T_max=n_epochs)

    if args.static:
        # optionally to switch to the static mode, which can bring speedup
        # on training
        model.q_layer.static_on(wires_per_block=args.wires_per_block)

    for epoch in range(1, n_epochs + 1):
        # train
        print(f"Epoch {epoch}:")
        train(dataflow, model, device, optimizer)
        print(optimizer.param_groups[0]["lr"])

        # valid
        valid_test(dataflow, "valid", model, device)
        scheduler.step()

    # test
    valid_test(dataflow, "test", model, device, qiskit=False)

    # run on Qiskit simulator and real Quantum Computers
    try:
        from qiskit import IBMQ
        from torchquantum.plugins import QiskitProcessor

        # firstly perform simulate
        print(f"\nTest with Qiskit Simulator")
        processor_simulation = QiskitProcessor(use_real_qc=False)
        model.set_qiskit_processor(processor_simulation)
        valid_test(dataflow, "test", model, device, qiskit=True)

        # then try to run on REAL QC
        backend_name = "ibmq_lima"
        print(f"\nTest on Real Quantum Computer {backend_name}")
        # Please specify your own hub group and project if you have the
        # IBMQ premium plan to access more machines.
        processor_real_qc = QiskitProcessor(
            use_real_qc=True,
            backend_name=backend_name,
            hub="ibm-q",
            group="open",
            project="main",
        )
        model.set_qiskit_processor(processor_real_qc)
        valid_test(dataflow, "test", model, device, qiskit=True)
    except ImportError:
        print(
            "Please install qiskit, create an IBM Q Experience Account and "
            "save the account token according to the instruction at "
            "'https://github.com/Qiskit/qiskit-ibmq-provider', "
            "then try again."
        )

if __name__ == "__main__":
    main()

Traceback (most recent call last):
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1194, in run_node
    return node.target(*args, **kwargs)
RuntimeError: The tensor has a non-zero number of elements, but its data is not allocated yet. Caffe2 uses a lazy allocation, so you will need to call mutable_data() or raw_mutable_data() to actually allocate memory.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1152, in get_fake_value
    return wrap_fake_exception(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 808, in wrap_fake_exception
    return fn()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1153, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1206, in run_node
    raise RuntimeError(
RuntimeError: Failed running call_function <built-in method tensor of type object at 0x7f82e375b500>(*([FakeTensor(FakeTensor(..., device='meta', size=(), dtype=torch.int64), cpu), FakeTensor(FakeTensor(..., device='meta', size=(), dtype=torch.int64), cpu)],), **{'dtype': torch.float32}):
The tensor has a non-zero number of elements, but its data is not allocated yet. Caffe2 uses a lazy allocation, so you will need to call mutable_data() or raw_mutable_data() to actually allocate memory.
(scroll up for backtrace)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 262, in <module>
    main()
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 217, in main
    train(dataflow, model, device, optimizer)
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 122, in train
    outputs = model(inputs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 82, in forward
    return self.dynamo_ctx(self._orig_mod.forward)(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 209, in _fn
    return fn(*args, **kwargs)
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 68, in forward
    qdev = tq.QuantumDevice(
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 106, in <graph break in forward>
    self.encoder(qdev, x)
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 108, in <graph break in forward>
    self.q_layer(qdev)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/yaoyao/repos/hidet/experiments/issue-265/main.py", line 40, in forward
    self.random_layer(qdev)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torchquantum/graph.py", line 25, in forward_register_graph
    res = f(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torchquantum/layers.py", line 240, in forward
    op(q_device)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 337, in catch_errors
    return callback(frame, cache_size, hooks)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 404, in _convert_frame
    result = inner_convert(frame, cache_size, hooks)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 104, in _fn
    return fn(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 262, in _convert_frame_assert
    return _compile(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 324, in _compile
    out_code = transform_code_object(code, transform)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 445, in transform_code_object
    transformations(instructions, code_options)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 311, in transform
    tracer.run()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1726, in run
    super().run()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
    and self.step()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
    getattr(self, inst.opname)(inst)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
    return inner_fn(self, inst)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
    self.call_function(fn, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 291, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 259, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 92, in call_function
    return tx.inline_user_function_return(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
    result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
    tracer.run()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
    and self.step()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
    getattr(self, inst.opname)(inst)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
    return inner_fn(self, inst)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
    self.call_function(fn, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 259, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 92, in call_function
    return tx.inline_user_function_return(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
    result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
    tracer.run()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
    and self.step()
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
    getattr(self, inst.opname)(inst)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
    return inner_fn(self, inst)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
    self.call_function(fn, args, kwargs)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/torch.py", line 548, in call_function
    tensor_variable = wrap_fx_proxy(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 754, in wrap_fx_proxy
    return wrap_fx_proxy_cls(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 789, in wrap_fx_proxy_cls
    example_value = get_fake_value(proxy.node, tx)
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1173, in get_fake_value
    raise TorchRuntimeError() from e
torch._dynamo.exc.TorchRuntimeError:

from user code:
   File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torchquantum/functional.py", line 1659, in cnot
    gate_wrapper(
  File "/home/yaoyao/miniconda3/lib/python3.8/site-packages/torchquantum/functional.py", line 258, in gate_wrapper
    params = torch.tensor(params, dtype=F_DTYPE)

Set torch._dynamo.config.verbose=True for more information

You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True

Hi @yaoyaoding,
thanks for adding the operators. I will further debug the new errors.
When you see there are lots of fusion opportunities do you mean even w/o complete complex dtype support once the new errors/bugs are fixed, we should see a noticeable speedup with hidet with the above code? (Of course this is empirical, just asking do you mean this would be the case in principle?)
Thanks again.

Hi @wcqc,
That depends. I am not familar the commonly used operators in the quantum networks and what operators are the bottleneck. Usually, fusion can greatly reduce the memory access and speedup your network. But if the bottleneck is on the operator like large matrix multiplication, then it will diminish the speedup of fusion other small operators (say, 20% of time for those small operators, you can only get at most 1/0.8 speedup even you can optimize the 20% to zero).
Thanks for trying hidet, and let me know if you find out how to fix/avoid above errors  (e.g., write the pytorch program in another way).


🤯 Cognitarium: Prevent blank nodes conflicts

When inserting triples, avoid same blank nodes being reused across insertions.
Issue description
Blank nodes allowing to create edges in graphs at insertion, their meaning is only linked to the set of triples being inserted and shall be ensured unique per insertion. Allowing to define same blank nodes across multiples insertions could affect previous ones and therefore alter unintentionally the data contained.
When inserting blank nodes, they shall be ensured unique scoped per insert message execution.
Proposal
In order to provide unique blank nodes at insertion, their identifier shall be formed with the context of this particular execution. I propose so to prefix alter their ids before writing by prefixing them with a hash representing the execution context, which shall carry the block height, the index of the transaction the message come from, and the insert message itself.

I complement what I've commented here: https://github.com/okp4/contracts/issues/494#issuecomment-1964395685
I agree with defining a method for generating blank node identifiers that ensures: uniqueness,determinism and efficiency.
Yet, I have reservations about employing hashing for this purpose. Why not opt for a straightforward counter for blank nodes, using a uint64? Since there is no need to reuse identifiers once they are freed, this approach seems more straightforward. It would be more efficient, require less computational effort, and use less storage space (8 bytes for a uint64 versus 32 bytes for a sha256 hash).

@amimart yes that's perfect. :relieved:


"Cannot write to HID device" after reopening Bitcoin app

I'm able to get the public key, however, it no longer works if I quit the Bitcoin app and reopen it.

Plug in Ledger Nano S
Enter PIN
Select "Bitcoin" app
Get the public keybtc.getWalletPublicKey_async('0')
// { publicKey, chainCode, bitcoinAddress }

Quit the Bitcoin app
Reopen the Bitcoin app
Get the public key againbtc.getWalletPublicKey_async('0')
// Error: Cannot write to HID device (ledger-comm-node.js:171)

Any help to get me pointed in the right direction is much appreciated. Thanks!

I figured out you have to "close" the device after quitting the app:
comm.device.close()


VSCodium

Make easily available on VSCodium too.

upvote!

Finally getting round to this, I am just awaiting approval for my namespace, and then the CI should be able to rerun and publish.


Updated opps-manual-setup-tutorial

Made changes to include coupon management service.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Expected the # of transactions &  CLV etc

Hi,
I have the real e-commerce data. Firstly, I've used "lifetimes" library for expected transactions, and expected average revenue, and CLV prediction. However, the model overfits and it fails when it comes to test data. There is a seriuos mismatch with the actual data and predicted data.
Then, I've found "Pymc".
I've transaction data and required information like you have in your tutorials. I've used Pymc bg nbd and gamma gamma models.
I've split the data for training and testing. I fit the model with the training data and then I predict the testing data and compared with the current actual data. The result doesn't satisfy. What to do to increase the accuracy? Any trick to validate the results, etc?
Which points do I miss?

Hey! Have you tried the Pareto/NB model (see https://www.pymc-marketing.io/en/stable/notebooks/clv/pareto_nbd.html) ? This model allows for time-invariant covariates, this might help the model performance.

Hey @sevimcengiz,
I presume you're working with this dataset:
https://www.kaggle.com/datasets/ankitverma2010/ecommerce-customer-churn-analysis-and-prediction/data
Databricks uses this same dataset in a notebook utilizing the btyd library, a fork I created of lifetimes before joining the pymc-marketing project:
https://notebooks.databricks.com/notebooks/RCG/Customer_Lifetime_Value/index.html#Customer_Lifetime_Value_1.html
The plots in that notebook suggest a wildly disparate population of customers. It's best to filter out extraneous customers when identified to improve performance.
The choice of time period on which to do the train/test split is also important due to the possibility of data drift over time. The same concept also makes rfm_train_test_split quite useful for evaluating the impact of a marketing campaign. Due to the difficulties in selecting an appropriate train/test split, you may wish to do a posterior predictive check instead (more on this at the end of this message).
It's also best to evaluate these models in aggregate because historical purchase frequency is given in integers, but predicted purchases are provided as decimal floats over time. Applying a discount rate with expected_customer_lifetime_value will also distort results further. Segmenting customers and evaluating performance by segment usually works well (the clv.rfm_segments utility was recently added and will be included in the next release).

Hey! Have you tried the Pareto/NB model (see https://www.pymc-marketing.io/en/stable/notebooks/clv/pareto_nbd.html) ? This model allows for time-invariant covariates, this might help the model performance.

Yes - not only does the ParetoNBDModel support covariates (and there are lot in this dataset!) but it also supports prior/posterior predictive checks, which is a more appropriate way to evaluate these models. We are still working on adding this functionality to the BetaGeoModel.

Thanks @juanitorduz, I'll implement it.
@ColtAllen thanks for the long and detailed answer. The databricks link helped me a a lot. The data that I'm working is real data but similar to the online retail data, I've quickly implemented all parts from the data bricks. I've followed each step but get "NaN" as a CLV value for all customers.

Firstly, why is this "purchases in the calibration period" written as a x-axis title name in the first figure? I think it should be "purchases in the holdout period". As I'm keeping the holdout period for the validation check. Model calculates and validates averagely. It doesn't fit perfectly but accaptable at the first point but getting "NaN" CLV for all customers makes me "hmm, which points I'm missing?".

I've followed each step but get "NaN" as a CLV value for all customers.

This is a common bug in lifetimes. Is this also happening in pymc-marketing? The plots you shared are all lifetimes functions.

Hi @sevimcengiz
Do you know if this is also an issue coming from pymc-marketing or just from lifetimes?

I faced with NaN when I used pymc.

@sevimcengiz  the API for these models has changed since this issue was created. Do you have any code examples you're able to share?

Actually, i haven’t check the code the last 5 months. But, I can check.Sevim CengizOn Nov 15, 2024, at 12:18 AM, Colt Allen @.***> wrote:
@sevimcengiz  the API for these models has changed since this issue was created. Do you have any code examples you're able to share?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>


What should get_obs_surface do when there's no results?

At the moment it raises a ValueError which I feel might be a bit much. Return None?

Let's keep this as raising a ValueError - that ties in better to the "programmatic view" of this function (compared to search). For the search function this shouldn't return a ValueError but just a None type ouput (e.g. still SearchResults object but prints a None or something).
Just need to make sure the error message is informative.


Defect: OpenCoarrays 2.10.0-11-gdfde1b9 and mpich release 4.0.2: 94% tests passed, 5 tests failed out of 85

The title of the issue should start with Defect: followed by a
succinct title.
Please make sure to put any logs, terminal output, or code in
fenced code blocks. Please also read the contributing guidelines
before submitting a new issue.
Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information.

[ ] I am reporting a bug others will be able to reproduce and not asking a question or requesting a new feature.

System information including:

OpenCoarrays Version: <!-- caf --version or ./install.sh --version -->
Fortran Compiler:
C compiler used for building lib:
Installation method:
All flags & options passed to the installer
Output of uname -a:
MPI library being used:
Machine architecture and number of physical cores:
Version of CMake:

To help us debug your issue please explain:
What you were trying to do (and why)
What happened (include command output, screenshots, logs, etc.)
What you expected to happen
Step-by-step reproduction instructions to reproduce the error/bug

Crashed edition.


Change the UI Mock table to use v-data-table instead of v-simple-table.

I am not sure if we actually want to do that. v-data-table does have pagination, but it doesn't allow buttons in each row from what I saw.

I have added pagination using v-pagination instead. But, I wouldn't be surprised if we decided to switch to v-data-table instead at some point so that sorting (by selected column) or something else is implemented for us.


Wildcard certificate dosn't protect base domain

A wildcard certificate for *.example.com is not valid for example.com. You need to add that name as well by specifying two -d flags to Certbot like:
./certbot-auto --server https://acme-v02.api.letsencrypt.org/directory -d "yii2.cc" -d "*.yii2.cc" --preferred-challenges=dns --manual certonly
That will give you a certificate with two names: yii2.cc and *.yii2.cc that will be valid for both the base domain and any first level subdomain.

Thanks, but this is not an an issue or bug related to certes, unless there is something wrong with documentation somewhere.

yes it is not a bug
can we have something to protect base domain at the same time ?
thank you

Yes you write that yourself, just add all the names you need to your order. You will then need to complete authorizations for each identifier before you can finalize your order and download your cert.
var certificateIdentifiers = new List<Identifier> {
          new Identifier { Type = IdentifierType.Dns, Value = "*.example.com" },
          new Identifier { Type = IdentifierType.Dns, Value = "example.com" }
      };

 order = await _acme.NewOrder(certificateIdentifiers);

works fine now
Thank You


o2r ERC unpacker for ReproZip

This is just to collect some ideas of an offline discussion on a potential unpacker into ERC. It would sort of entail the Docker unpacker, but elicitate additional metadata from the user (with a nice CLI UI?) which ReproZip would normally not capture, such as author information, the main document and view file, etc. Before I even think about implementing such a thing, some questions:
@VickySteeves IMHO this is also a nice way to learn how the approaches and goals of o2r and ReproZip differ :-)
@remram44 We want to capture two aspects of the runtime environment, an image and a manifest, the latter is usually the Dockerfile for us. I'm wondering if you have an idea what kind of manifest ReproZip could produce... maybe this is the log of all traced files? Maybe it is the output of dpkg --list filtered by the libs which are traced (sounds tricky)?

ReproZip already does the tricky dpkg (and rpm) integration, capturing which DEB or RPM packages the traced files come from. But more could be done: Ruby/R/Python packages could be inferred in a similar way (which could be a base for your workspace-to-Docker process).
Note that ReproZip has support for plugins, that could collect additional information at packing time. It is only currently used to filter input files a bit, but for example, I envision looking for Git repositories at some point, so version information can be captured (only works if running from a source tree).


Publishing to Spark Packages

Hi,
I noticed that you ran into several problems while publishing a release to Spark Packages. I strongly suggest that you use the sbt-spark-package plugin, as it would greatly ease your publishing process. The Spark Packages repo has several different requirements regarding publishing, which you can find here. Please let me know if you have any questions, and we would greatly appreciate any feedback you might have!
Thanks,
Burak

Thanks for the pointer, I'll keep track of the plugin! :)


Fix strict variables

Fix strict variables for defaults in params.pp
Fix versioncmp for Puppet 4 in params.pp
Test for undefined variable in snmp.conf.erb
Add missing trap_service_name for Debian in params.pp

Fixes Travis errors from #97

@gusnuf @razorsedge refactored to comply with CONTRIBUTING.  I will re-submit another PR for the master/agentx changes once this PR is merged.

If support for top-scope variables was no longer needed, something like #85 would make much more sense since it makes params.pp much cleaner.  If that were the case, I would rebase this and just fix the RedHat-specific variables and the versioncmp issue in params.pp.

@razorsedge I believe I answered everything inline...

@razorsedge should be all set.  Set it to empty string instead of snmptrapd

empty string 👎 , undef 👍


Add travis-ci testing

Added a simple config for Travis-Ci that runs tests under Lua 5.1-5.3 and LuaJIT 2.0 using their container infrastructure (no sudo used). You'll still have to log into travis-ci site and enable testing for this repo, but after that tests will be executed automatically for each push, like this.

Thanks, this is an excellent incentive for getting as many contributed tests as possible.

Yep, I'm green! This is indeed cool.


taskadel not accepting user arguments

I noticed when running this, we can't seem to get !taskadel to notice a recipient. At first I thought I was just using the command wrong, however further testing identified that the command doesn't accept any arguments.
For example, we have two accounts, userA a moderator and userB a viewer. userA creates a task, userB also creates a task. Let's say userA now wants to delete userB's task, you would expect to run !taskadel userB, however that deletes userA's task rather than userB's task. You run the command again, noticing that userA's task is gone, and the message returned tells you that userA has no tasks.

@reddit-sysalex I will look into it right away
Just to double check you are using the latest version of the task list correct?

@mohamed-tayeh Thanks a lot. Correct, using the latest version.


Onboarding with Mattermost Blog Post

Summary
Blog post about my onboarding with Mattermost
Ticket Link
https://mattermost.atlassian.net/browse/MM-21130

Thanks @amyblais and @jasonblais for the review! I loved the feedback and made the updates, thank you for the keen eye!

I'm wondering if @justinegeffen should glance over this. She did review the other recent blog posts and did a great job doing so.
Thoughts?

I'm wondering if @justinegeffen should glance over this. She did review the other recent blog posts and did a great job doing so.
Thoughts?

Thanks for the tag! I fixed a few minor typos/punctuation errors but otherwise it's a great blog post. :)

I'm wondering if @justinegeffen should glance over this. She did review the other recent blog posts and did a great job doing so.
Thoughts?

Thanks for the tag! I fixed a few minor typos/punctuation errors but otherwise it's a great blog post. :)

Thanks @justinegeffen, I've updated the noted items!

I think this one looks ready to merge then :) :tada:


Dataset is non-manifold and cannot be subdivided.

Hi @marcomusy,
I am having the following point clouds (attached files) which I would like to further subdivide.
import os
from glob import glob
import natsort
import vedo as vd

if __name__ == '__main__':

    pcdFiles = natsort.natsorted(glob('**/*.ply', recursive=True))

    for i, pcdFile in enumerate(pcdFiles):
        print(pcdFile)
        s = vd.Mesh(pcdFile).alpha(0.2)
        m = s.subdivide(method=1, N=3)
        # vd.show(m, axes=8)

I tried to use methods 0 or 1 but I am getting the following errors respectively:
For method 0:
airplane_0001.ply
2021-09-27 16:34:47.784 (   0.389s) [        96BBA740]vtkLoopSubdivisionFilte:149   WARN| vtkLoopSubdivisionFilter (0x55b145a98af0): numCellsInLoop < 1: 0
2021-09-27 16:34:47.786 (   0.390s) [        96BBA740]vtkApproximatingSubdivi:112    ERR| vtkLoopSubdivisionFilter (0x55b145a98af0): Subdivision failed.
2021-09-27 16:34:47.786 (   0.390s) [        96BBA740]       vtkExecutive.cxx:753    ERR| vtkCompositeDataPipeline (0x55b145b2a4e0): Algorithm vtkLoopSubdivisionFilter(0x55b145a98af0) returned failure for request: vtkInformation (0x55b1456156a0)
  Debug: Off
  Modified Time: 210649
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA
  FROM_OUTPUT_PORT: 0
  ALGORITHM_AFTER_FORWARD: 1
  FORWARD_DIRECTION: 0

airplane_0002.ply
2021-09-27 16:34:47.816 (   0.420s) [        96BBA740]vtkLoopSubdivisionFilte:149   WARN| vtkLoopSubdivisionFilter (0x55b1462a5350): numCellsInLoop < 1: 0
2021-09-27 16:34:47.818 (   0.422s) [        96BBA740]vtkApproximatingSubdivi:112    ERR| vtkLoopSubdivisionFilter (0x55b1462a5350): Subdivision failed.
2021-09-27 16:34:47.818 (   0.422s) [        96BBA740]       vtkExecutive.cxx:753    ERR| vtkCompositeDataPipeline (0x55b145615760): Algorithm vtkLoopSubdivisionFilter(0x55b1462a5350) returned failure for request: vtkInformation (0x55b146183e60)
  Debug: Off
  Modified Time: 449068
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA
  FROM_OUTPUT_PORT: 0
  ALGORITHM_AFTER_FORWARD: 1
  FORWARD_DIRECTION: 0

airplane_0003.ply
2021-09-27 16:34:47.839 (   0.444s) [        96BBA740]vtkLoopSubdivisionFilte:149   WARN| vtkLoopSubdivisionFilter (0x55b1462c1080): numCellsInLoop < 1: 0
2021-09-27 16:34:47.840 (   0.444s) [        96BBA740]vtkApproximatingSubdivi:112    ERR| vtkLoopSubdivisionFilter (0x55b1462c1080): Subdivision failed.
2021-09-27 16:34:47.840 (   0.444s) [        96BBA740]       vtkExecutive.cxx:753    ERR| vtkCompositeDataPipeline (0x55b1462a4b40): Algorithm vtkLoopSubdivisionFilter(0x55b1462c1080) returned failure for request: vtkInformation (0x55b1462be8a0)
  Debug: Off
  Modified Time: 539434
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA
  FROM_OUTPUT_PORT: 0
  ALGORITHM_AFTER_FORWARD: 1
  FORWARD_DIRECTION: 0

airplane_0004.ply
2021-09-27 16:34:47.862 (   0.466s) [        96BBA740]vtkLoopSubdivisionFilte:149   WARN| vtkLoopSubdivisionFilter (0x55b1462a2630): numCellsInLoop < 1: 0
2021-09-27 16:34:47.863 (   0.467s) [        96BBA740]vtkApproximatingSubdivi:112    ERR| vtkLoopSubdivisionFilter (0x55b1462a2630): Subdivision failed.
2021-09-27 16:34:47.863 (   0.467s) [        96BBA740]       vtkExecutive.cxx:753    ERR| vtkCompositeDataPipeline (0x55b145da4bd0): Algorithm vtkLoopSubdivisionFilter(0x55b1462a2630) returned failure for request: vtkInformation (0x55b1462a47d0)
  Debug: Off
  Modified Time: 748860
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA
  FROM_OUTPUT_PORT: 0
  ALGORITHM_AFTER_FORWARD: 1
  FORWARD_DIRECTION: 0

For method 1:
airplane_0001.ply
airplane_0002.ply
2021-09-27 16:35:43.543 (   1.378s) [        38451740]vtkLinearSubdivisionFil:68     ERR| vtkLinearSubdivisionFilter (0x55e0a73d7340): Dataset is non-manifold and cannot be subdivided.
2021-09-27 16:35:43.549 (   1.385s) [        38451740]vtkInterpolatingSubdivi:97     ERR| vtkLinearSubdivisionFilter (0x55e0a73d7340): Subdivision failed.
2021-09-27 16:35:43.550 (   1.385s) [        38451740]       vtkExecutive.cxx:753    ERR| vtkCompositeDataPipeline (0x55e0a66ccf40): Algorithm vtkLinearSubdivisionFilter(0x55e0a73d7340) returned failure for request: vtkInformation (0x55e0a73ceb70)
  Debug: Off
  Modified Time: 1690347
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA
  FROM_OUTPUT_PORT: 0
  ALGORITHM_AFTER_FORWARD: 1
  FORWARD_DIRECTION: 0

airplane_0003.ply
airplane_0004.ply

Any idea what could be the problem and possibly solve it?
Thanks.
pcds.zip

Hi, if the mesh is non-manifold i'm not sure what one can do..
it seems you have lots of them:
from vedo import *

settings.useDepthPeeling = True

m = Mesh("data/airplane_0001.ply").alpha(0.2).lw(0.1)

non_mani = m.boundaries(boundaryEdges=False, nonManifoldEdges=True, returnCellIds=False)

non_mani_ids = m.boundaries(boundaryEdges=False, nonManifoldEdges=True, returnPointIds=True)
pts = Points(m.points(non_mani_ids))

show(non_mani, m, pts, axes=1)

Oohh I see, is there any function to repair/fix such meshes?

uhm I just tried pymeshfix... like in examples/other/remesh_meshfix.py but it doesn't work either..
from vedo import *
from pymeshfix import MeshFix

m = Mesh("data/airplane_0001.ply").alpha(0.2).lw(0.1)
meshfix = MeshFix(m.points(), m.faces())
meshfix.repair()
m = Mesh(meshfix.mesh)
show(m,  axes=1)

Thanks Marco,
I'll check if there is any other lib that might be useful to solve the non-manifold issue, otherwise I will see what I'll do.
I'll close this for now.

Just to mention, in case that someone stumbles upon this, that in case that the up-sampling is not mesh dependent someone can use the densify function to up sample points.


Decouple style from segment object

Resolves #567
Note: in the current state of this PR, I reverted the change in #578, which fixed #573. I converted the example code into a TestFX test, and it passes.

@afester @JFormDesigner any objections to or comments on this? Also, this PR isn't completely "clean" when it comes to its scope. I exposed some things or renamed methods that I later realized didn't need to be exposed/renamed.

I can confirm that #584 does not occur anymore with this PR, so :+1:  from my side

👍  I've tested this PR with Markdown Writer FX and it works without problems. Thanks.


trackStart event gets emitted twice for queue songs

When adding the first song of the queue the trackStart works perfectly while when like the second song of the queue starts the event gets emitted twice and idk why.
Here's the trackStart event i made:
manager.on("trackStart", async(player, track) => {
        nowPlaying.setDescription(`**Now playing**: ${ track.title } [${ getYoutubeLikeToDisplay(track.duration) }]`)
        await eventsChannel.send({embeds: [nowPlaying]})
    })

First, what's eventsChannel defined by? For myself it's fine.

use manager.once()


Update Meson and fix acquire meson for non-windows systems.

Update Meson to 0.47.1.
Fix acquire meson on Unix platforms.

vcpkg Issue here

Thanks for the PR!

The latest meson release since last week is 0.47.2, you might want to update to that.

در تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۶ ق.ظ، "Nirbheek Chauhan"<EMAIL_ADDRESS>نوشت:
The latest meson release since last week is 0.47.2, you might want to
update to that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188518, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlhT4lnAuXmWoQKOhXDvhQHp1FRqi_8Fks5uV24tgaJpZM4WLZNB
.

در تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۷ ق.ظ، "sojdemoghadam"<EMAIL_ADDRESS>نوشت:
در تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۶ ق.ظ، "Nirbheek Chauhan"<EMAIL_ADDRESS>نوشت:
The latest meson release since last week is 0.47.2, you might want to
update to that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188518, or
mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AlhT4lnAuXmWoQKOhXDvhQHp1FRqi_8Fks5uV24tgaJpZM4WLZNB

.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188605, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlhT4tKsPz7ALfhYBYmspBkcFQPvtm30ks5uV25egaJpZM4WLZNB
.

از گوشی هوشمند Samsung Galaxy ارسال شده است.
-------- پیام اصلی --------از: sojdemoghadam<EMAIL_ADDRESS>تاریخ: ۲۰۱۸/۸/۳۰  ۹:۱۸  (GMT+03:30) گیرنده: Microsoft/vcpkg<EMAIL_ADDRESS>گیرنده کپی: sojdemoghadam<EMAIL_ADDRESS>Your activity<EMAIL_ADDRESS>موضوع: Re: [Microsoft/vcpkg] Update Meson and fix acquire meson for non-windows systems. (#4155)
در تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۷ ق.ظ، "sojdemoghadam"<EMAIL_ADDRESS>نوشت:
در تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۶ ق.ظ، "Nirbheek Chauhan"<EMAIL_ADDRESS>نوشت:
The latest meson release since last week is 0.47.2, you might want to
update to that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188518, or
mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AlhT4lnAuXmWoQKOhXDvhQHp1FRqi_8Fks5uV24tgaJpZM4WLZNB

.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188605, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlhT4tKsPz7ALfhYBYmspBkcFQPvtm30ks5uV25egaJpZM4WLZNB
.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/Microsoft/vcpkg","title":"Microsoft/vcpkg","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/Microsoft/vcpkg"}},"updates":{"snippets":[{"icon":"PERSON","message":"@sojdemoghadam in #4155: در تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۷ ق.ظ، "sojdemoghadam" \u003cnotifications@github.com\u003e\nنوشت:\n\nدر تاریخ ۳۰ اوت ۲۰۱۸ ۹:۱۶ ق.ظ، "Nirbheek Chauhan" \u003cnotifications@github.com\u003e\nنوشت:\n\nThe latest meson release since last week is 0.47.2, you might want to\nupdate to that.\n\n—\nYou are receiving this because you are subscribed to this thread.\nReply to this email directly, view it on GitHub\n\u003chttps://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188518\u003e, or\nmute\nthe thread\n\u003c\nhttps://github.com/notifications/unsubscribe-auth/AlhT4lnAuXmWoQKOhXDvhQHp1FRqi_8Fks5uV24tgaJpZM4WLZNB\n\u003e\n\n.\n\n—\nYou are receiving this because you are subscribed to this thread.\nReply to this email directly, view it on GitHub\n\u003chttps://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188605\u003e, or mute\nthe thread\n\u003chttps://github.com/notifications/unsubscribe-auth/AlhT4tKsPz7ALfhYBYmspBkcFQPvtm30ks5uV25egaJpZM4WLZNB\u003e\n.\n"}],"action":{"name":"View Pull Request","url":"https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188680"}}}
[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188680",
"url": "https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188680",
"name": "View Pull Request"
},
"description": "View this Pull Request on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
},
{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"hideOriginalBody": "false",
"originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB",
"title": "Re: [Microsoft/vcpkg] Update Meson and fix acquire meson for non-windows systems. (#4155)",
"sections": [
{
"text": "",
"activityTitle": "sojdemoghadam",
"activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png",
"activitySubtitle": "@sojdemoghadam",
"facts": [
]
}
],
"potentialAction": [
{
"name": "Add a comment",
"@type": "ActionCard",
"inputs": [
{
"isMultiLine": true,
"@type": "TextInput",
"id": "IssueComment",
"isRequired": false
}
],
"actions": [
{
"name": "Comment",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n"commandName": "IssueComment",\n"repositoryFullName": "Microsoft/vcpkg",\n"issueId": 4155,\n"IssueComment": "{{IssueComment.value}}"\n}"
}
]
},
{
"targets": [
{
"os": "default",
"uri": "https://github.com/Microsoft/vcpkg/pull/4155#issuecomment-417188680"
}
],
"@type": "OpenUri",
"name": "View on GitHub"
},
{
"name": "Unsubscribe",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n"commandName": "MuteNotification",\n"threadId": 372085569\n}"
}
],
"themeColor": "26292E"
}
]


Change pushHash with stringified data

Notification with loc-key and loc-args crash because data.alert is not a string.
e.g.
alert: {
  'loc-key': '....',
  'loc-args': '...',
}

[Current coverage][cc-pull] is 75.44% (diff: 0.00%)

Merging [#2397][cc-pull] into [master][cc-base-branch] will decrease coverage by 16.39%

@@             master      #2397   diff @@
==========================================
  Files            95         95
  Lines         10708      10708
  Methods        1309       1309
  Messages          0          0
  Branches       1741       1741
==========================================
- Hits           9835       8079   -1756
- Misses          873       2629   +1756
  Partials          0          0

Powered by Codecov. Last update [fa49f05...0af60ab][cc-compare]
[cc-base-branch]: https://codecov.io/gh/ParsePlatform/parse-server/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/ParsePlatform/parse-server/compare/fa49f05fddbeecfc81dfa4506272ae375875bbcd...0af60ab693fca92be32322ba71c6fe459bc14292?src=pr
[cc-pull]: https://codecov.io/gh/ParsePlatform/parse-server/pull/2397?src=pr

@alenoir updated the pull request.

@alenoir updated the pull request.

@alenoir updated the pull request.

@alenoir updated the pull request.

as we said in https://github.com/ParsePlatform/parse-server/commit/676d2e28de1c335448d1c1a997371b011f9630e2 i replace payloadstring by stringify data.alert for localization format

the same for me.
info: warning: error while sending push TypeError: Not a string or buffer
and then
Error: {"code":141,"message":{"code":107,"message":"Received an error with invalid JSON from Parse: \r\n<title>502 Bad Gateway</title>\r\n\r\n
502 Bad Gateway
\r\n
nginx/1.6.2\r\n\r\n\r\n"}}

This should be merged soon! All iOS localized push will fail on version 2.2.17...

thanks @alenoir. Tests would have been nice too.

@alenoir updated the pull request.

Thanks @alenoir, merging after travis


TEXT column type is not supported (CLOB)

I got a problem. The column type TEXT (not varchar) on h2database is not supported by assertjdb too (It is mapped as object JdbcClob.class that implements the java.sql.CLOB).
Then I see that on AbstractDbData class is not mapped the CLOB type on the collectRowsFromResultSet method.
I suggest just add this type to get it as string as you can see below:
switch (type) {
        case Types.DATE:
          objectsList.add(resultSet.getDate(columnName));
          break;
        case Types.TIME:
          objectsList.add(resultSet.getTime(columnName));
          break;
        case Types.TIMESTAMP:
          objectsList.add(resultSet.getTimestamp(columnName));
          break;
        case Types.CLOB:
          objectsList.add(resultSet.getString(columnName));
          break;
        default:
          objectsList.add(resultSet.getObject(columnName));
          break;
        }

What do you think?

It is a good idea too.
I think there are other H2 SQL types with problems : BLOB and ARRAY

Ok I already made this and just need to create some tests too. I will create a pull request on another branch.
For the BLOB type I don't know if is better to get it as String or maybe bytes...

I think ARRAY type cannot be mapped as String too:
ARRAY http://h2database.com/html/datatypes.html#array_type
An array of values. Mapped to java.lang.Object[] (arrays of any non-primitive type are also supported).

I still think that can be very useful an assertion point to verify these unmapped types on assertjdb. Otherwise you'll need to implement every type support and it can be unnecessary. The only thing important on this cases is that the input type must be the same value of the expected type.

Yes i have an idea for unmapped types : http://joel-costigliola.github.io/assertj/assertj-core-conditions.html
Please let me time and i will talk with you about an idea.
If you are ok, you can finish your first pull request, after it is fine if you want to implements BLOB and CBLOB like for UUID.
For CBLOB (http://www.h2database.com/html/datatypes.html#clob_type), i think is possible to map with String
For BLOB (http://www.h2database.com/html/datatypes.html#blob_type), a byte[]
For ARRAY (http://www.h2database.com/html/datatypes.html#array_type), a Object[] so i think we need to take time to think about the way to do

https://github.com/joel-costigliola/assertj-db/pull/20

I add specific tests. Please let me monday evening and you can use that to help you for implementation of specific sql types.

I improve the test class "H2Database_Test".
I am sorry : it is finally different than a part of your pull request #20 (for the tests)
But i think it better like that because there are all the sql types of H2 and after we can add test on other specific database.
If you want and have time to fork from this point to implement and finish the test it is fine otherwise i can that tomorrow but now it is time to close the eyes ;-)

I closed my pull request about this issue because your implementation is already cover this.


MpConfig will never include Helidon Config property names in the return value of getPropertyNames()

MpConfig as a MicroProfile Config implementation acquires and caches all property names at construction time for returning later from its getPropertyNames() method.  This is an area of the MicroProfile Config specification that is undefined and untested by the TCK (must it cache? must it not cache? is caching permitted? prohibited?) so this is probably fine in itself.
To do so, it interrogates its MicroProfile ConfigSources for their property names, and concatenates that set together with what is hoped is the full key names of all leaf nodes in a Helidon Config tree.  The combined set should represent all property names known to the MpConfig instance at construction time.
(You can see where application.yaml is added as a Helidon configuration source after all the MicroProfile ConfigSources here:
https://github.com/oracle/helidon/blob/173f2c07161c27f4548a77ec37ec154e0c6e0160/microprofile/config/config/src/main/java/io/helidon/microprofile/config/MpConfigBuilder.java#L143-L147
)
The MicroProfile ConfigSource interrogation works fine.  The Helidon interrogation does not; its code can be found here:
https://github.com/oracle/helidon/blob/173f2c07161c27f4548a77ec37ec154e0c6e0160/microprofile/config/config/src/main/java/io/helidon/microprofile/config/MpConfig.java#L78-L80
What I think is happening is that the Predicate that is supplied to traverse is Config::isLeaf, which I think returns false for the root node of the tree, and hence it appears that that node and all of its descendants are ignored.
Another way to put this is when that predicate is Config::isLeaf, and traverse is supplied with it, then traverseSubNodes is never entered in the code excerpt below, even when, for example, application.yaml containing valid YAML properties is found and loaded by the Helidon Config subsystem:
https://github.com/oracle/helidon/blob/173f2c07161c27f4548a77ec37ec154e0c6e0160/config/config/src/main/java/io/helidon/config/ConfigComplexImpl.java#L57-L79

Replacing MpConfig's recipe for acquiring property names with the following fixes the problem:
this.propertyNames =
                Stream.concat(mpConfigSources.stream()
                                      .flatMap(cs -> cs.getPropertyNames().stream()),
                              config.traverse()
                              .filter(Config::isLeaf)
                              .map(Config::key)
                              .map(Config.Key::toString))
            .collect(Collectors.toSet());


Selecting a kernel from the marketplace does not show other notebooks

Originally filed in https://github.com/microsoft/vscode/issues/129652.
When opening a .NET notebook for the first time and selecting a kernel, you are always shown the Jupyter extension and there is no clear guidance about needing the .NET Interactive Notebook. It would be great if VS Code could detect the various languages used in a notebook and then show a list of relevant notebooks.
@ 00:05 is where I select a kernel
https://user-images.githubusercontent.com/35271042/135929628-21320177-a40a-4350-9976-9e121aebc7a3.mp4

If you run a cell from a .ipynb polyglot notebook and don't have any notebook extensions installed, then you get a prompt to search @tag:notebookKernelJupyterNotebook in the marketplace. That just finds the Jupyter extensions. Should the Polyglot extension also have that tag @rebornix?

@roblourens Are these tags documented?

Not that I know, @rebornix?

@DonJayamanne they are documented here: https://code.visualstudio.com/api/extension-guides/notebook#controller

I think that doc should say why you should do that

Updated the docs https://github.com/microsoft/vscode-docs/commit/244fd9c421faa6c3821001dfe46a016a56703ef0


RFC 5054 paddings

RFC 5054 specifies left padding during calculation of k and u as expressed below:
k = H(N, PAD(g))
u = H(PAD(A), PAD(B))
PAD(X) inserts left padding (zeros) into the binary big-endian representation of X to match it's number of bytes with the number of bytes in N.
Current code doesn't compute the paddings.

You're correct about that. CSRP was written before RFC5054 became the
de-facto standard for inter-implementation padding should be implemented
and, unfortunately, changing the default implementation would break
existing applications that are already dependent on the current hashing
approach. That said, there is an rfc5054_compat branch of csrp that
implements rfc5054 and is known to be compatible with other libraries. It
should be relatively straight-forward to modify the default csrp
implementation to default to the current hashing approach but be easily
converted to rfc5054 compliance via a a command-line argument but doing so
hasn't yet been a high enough priority for someone to submit a patch for
it. I've received several notes similar to yours though so I think it's
time to update the readme to explicitly call this out and point people over
to the rfc5054_compat branch.
On Sat, Feb 10, 2018 at 9:18 AM, amitkriit<EMAIL_ADDRESS>wrote:

RFC 5054 specifies left padding during calculation of k and u as
expressed below:
k = H(N, PAD(g))
u = H(PAD(A), PAD(B))
PAD(X) inserts left padding (zeros) into the binary big-endian
representation of X to match it's number of bytes with the number of bytes
in N.
Current code doesn't compute the paddings.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cocagne/csrp/issues/9, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABrLfBAg74CfCAdJx4nOZbz91ngzERMaks5tTbM5gaJpZM4SA9hp
.


[IPv6 upstream support] Error while accessing the  upstream services which resolve to IPv6 addresses

Hi,
I am trying to bring up Kong in a a kuberentes environment where  the upstream services resolve to IPv6 IP addresses (and the DNS servers use IPv6 addresses)
My kong version is 0.14.1.
When i try to resolve the upstream services, i am getting the following error:

2019/10/21 08:11:27 [error] 330#0: *67 [lua] balancer.lua:806: execute(): [dns] dns lookup pool exceeded retries (1): failed to create a resolver: failed to set peer name: network is unreachable. Tried: (short)mgmt.system.svc.cluster.local:(na) - cache-miss
mgmt.system.svc.cluster.local:33 - cache-miss/scheduled/try 1 error: failed to create a resolver: failed to set peer name: network is unreachable/scheduled/try 2 error: failed to create a resolver: failed to set peer name: network is unreachable/dns lookup pool exceeded retries (1): failed to create a resolver: failed to set peer name: network is unreachable
mgmt.system.svc.cluster.local.system.svc.cluster.local:33 - cache-miss/scheduled/try 1 error: failed to create a resolver: failed to set peer name: network is unreachable/scheduled/try 2 error: failed to create a resolver: failed to set peer name: network is unreachable/dns lookup pool exceeded retries (1): failed to create a resolver: failed to set peer name: network is unreachable
mgmt.system.svc.cluster.local.svc.cluster.local:33 - cache-miss/scheduled/try 1 error: failed to create a resolver: failed to set peer name: network is unreachable/scheduled/try 2 error: failed to create a resolver: failed to set peer name: network is unreachable/dns lookup pool exceeded retries (1): failed to create a resolver: failed to set peer name: network is unreachable
mgmt.system.svc.cluster.local.cluster.local:33 - cache-miss/scheduled/try 1 error: failed to create a resolver: failed to set peer name: network is unreachable/scheduled/try 2 error: failed to create a resolver: failed to set peer name: network is unreachable/dns lookup pool exceeded retries (1): failed to create a resolver: failed to set peer name: network is unreachable

This issue looks to be similar to https://github.com/Kong/kong/issues/3838

Hello,
Looks like your machine doesn't have IPv6 connectivity to the upstream. Can you confirm that please?
See: failed to set peer name: network is unreachable means Kong tried to connect but O/S reported network being unreachable.
Datong

@prabhakar-pal  What changes did you do to make possible Kong talking to Cassandra over IPv6?
I have the same problem and I'm looking for the solution.
Best Regards,
Ania

That version of OpenResty (or even earlier ones) should have IPv6 support enabled. Are you running Kong inside a container or something similar? I am thinking of reasons why IPv6 routes does not exists for Kong's execution environment here.


fix: #110 package.patterns

Update to latest @types/serverless which supports package.patterns
Updated README to mention patterns and updated link to serverless docs
Uses package.patterns instead of patterns.include in the global package section
Added support for package.patterns on individual functions. It does this by copying the individual files into a special folder in the build folder, which gets ignored by all other functions. Then that special folder prefix gets removed when the intended function is packaged.

I'm not sure it's the best way to solve but at least it's a way to solve it :)

Hi @kabo and thank you for this pull request. I will check it later today or in the next days, but on top of my head I think you disabled include/exclude support, however we cannot assume what version of serverless people are using, especially since deprecation dates from last month. This would be a breaking change for serverless-esbuild. So I think first thing would be to support both include/exclude and patterns options.

Cool, yeah if the approach looks good otherwise I can look into that.

Cool. There is some other stuff that will need addressing, in how the flow works and file get copied. But I don't have time just right now to check that up, so I'll comment once I've done a deeper review, especially concerning edge cases.

Hey, so I went a bit deeper on the review. Seems to me that there's still a couple of pieces missing: in global packaging, the patterns of each function are simply ignored. Also, https://github.com/floydspace/serverless-esbuild/blob/353ecd5a6d3840af62b97997405fcc66703cce41/src/pack-externals.ts#L179 uses the exclude props this has not been updated.
More generally I am not a huge fan of the prefixed directory. I mean, it could work like that (once addressed the global packaging issue) but i'd rather have some more "data" driven approach. It's just a sketch in my head now, but we might be better off copying all the included files from global and functions to the build directory, and then use the complete pattern list (inclusion and exclusion) when copying to the zip file? Or only work with includes and treat excludes only in the dependencies level, which is what I think we are doing now? Am I making enough sense?
In this view, it might be better to distribute the patterns to include and exclude rather than the contrary? But this is still rough thinking.
On a side note @floydspace, I see that we use glob and globby. We might just stick to globby everywhere? that would save us a stat directory lookup in the packing phase (because globby have an option to output files only) and could speedup build as globby uses fast-glob which sounds faster than glob (am I falling for the name here ?)

in global packaging, the patterns of each function are simply ignored
hmm, yeah, didn't think of that scenario.

But buildOptions.exclude is correct right? serverless has service.package.patterns and function.package.patterns, buildOptions seems to be populated from custom.esbuild, which to my understanding hasn't changed. From the README:

The package specified in the exclude option is passed to esbuild as external, but it is not included in the function bundle either.

I don't see a reason to mess with that.

Regarding include/exclude vs patterns my thinking was the include/exclude pattern has been deprecated in favour of patterns. This suggests to me that the future proof way of doing it is patterns, otherwise in the future we'll be stuck with our own include/exclude pattern that isn't being used anywhere else, it is then a custom pattern that maps over the pattern serverless is using and necessitates converting between the two. Sure, if we still want to support include/exclude we'll have to convert that to patterns, but eventually we should be able drop include/exclude and just remove that code. This was my reasoning.

I have tested this code with:

individual patterns, individual packaging
individual patterns, no individual packaging
global patterns, no individual packaging
global patterns, individual packaging

But buildOptions.exclude is correct right? serverless has service.package.patterns and function.package.patterns, buildOptions seems to be populated from custom.esbuild, which to my understanding hasn't changed. From the README:

The package specified in the exclude option is passed to esbuild as external, but it is not included in the function bundle either.
Yes actually you are right

Regarding include/exclude vs patterns my thinking was the include/exclude pattern has been deprecated in favour of patterns. This suggests to me that the future proof way of doing it is patterns, otherwise in the future we'll be stuck with our own include/exclude pattern that isn't being used anywhere else, it is then a custom pattern that maps over the pattern serverless is using and necessitates converting between the two. Sure, if we still want to support include/exclude we'll have to convert that to patterns, but eventually we should be able drop include/exclude and just remove that code. This was my reasoning.

Yes, this is right. I was more thinking of a two steps mechanism of inclusion-then-inclusion/exclusion per function based on the option declaration rather than relying on a filesystem distribution. But If you have that tested and working, I believe it's pretty good enough.

I see that we use glob and globby. We might just stick to globby everywhere? that would save us a stat directory lookup in the packing phase (because globby have an option to output files only) and could speedup build as globby uses fast-glob which sounds faster than glob

Works for me.

Given this setup in serverless.yml
funcA:
  handler: src/funcA.handler
  package:
    patterns:
    - resources/some.txt
funcB:
  handler: src/funcB.handler

You'd end up with a build folder like this:

__only_funcA/resources/some.txt
funcA.js
funcB.js

Then when it's zipped:

funcA.zip containing resources/some.txt and funcA.js
funcB.zip containing funcB.js

Hope that clarifies.


Basic auth instead of OAuth

Hi,
Alerta seems to be a great tool and exactly what I'm looking for (Aggregate Zabbix and PingDom)
But we don't want to use external OAuth service and don't want to implement it internally just for alerta.
Is it possible to just have a basic user/password auth ?
As this service won't be exposed on the internet I have no need to couple it to an Exernal Auth mechanism.
Thanks

Or at least provide a custom oauth to get it work on a in-house oauth platform

Alerta does not currently support basic auth or a custom oauth platform. However, the client-side auth library is satellizer[1] and it does support either of those options so the building blocks are there.
[1] https://github.com/sahat/satellizer

Fixed by #87

That's great thanks

I tried to build it but I'm not sure which alertad.conf setup I have to do.
I kept AUTH_REQUIRED = True but I don't know how to manage that.
Do I have to set a specific parameter ? ALLOWED_EMAIL_DOMAINS for example ?
Thanks

Get alerta version 4.3.0 and the latest version of the web UI. You only need to set AUTH_REQUIRED = True in alertad.conf and check web UI config.js has provider set to basic. All other settings are fine by default.


Retrieve/display mavlink messages

Basic requirement to display any mavlink messages, such as reasons for not arming, autotune start/stop etc

Added graphql StatusText message support.  Added a 'snackbar' (dropdown from top alert that disappears after 6 seconds) to display incoming messages.  Also added a warning icon with a dropdown of historical messages like gqroundcontrol, and delete button to clear messages.