Error in Logger on epoch end when using Multiple GPUs

🐛 Bug
When using multiple GPUs with 'dp', the error RuntimeError: All input tensors must be on the same device. Received cuda:1 and cuda:0 occurs. It means the collections on epoch end would be from different device.
Expected behavior
While they might need to be on the same device, or maybe the aggregating function should be able to handle items from different device.
Environment

PyTorch Version (e.g., 1.0): 1.7.0
OS (e.g., Linux): Linux
How you installed PyTorch (conda, pip, source): pip
Python version: 3.8.0
CUDA/cuDNN version: 10.2
Any other relevant information: pytorch-lightning==1.0.8

A quick but not safe solution

modify collate_tensors function in pytorch_lightning/core/step_result.py:

def collate_tensors(items: Union[List, Tuple]) -> Union[Tensor, List, Tuple]:

    if not items or not isinstance(items, (list, tuple)) or any(not isinstance(item, Tensor) for item in items):
        # items is not a sequence, empty, or contains non-tensors
        return items

    # add the following line of code
    items = [item.type_as(items[0]) for item in items]

    if all(item.ndim == 0 for item in items):
        # all tensors are scalars, we need to stack
        return torch.stack(items)

    if all(item.ndim >= 1 and item.shape[1:] == items[0].shape[1:] for item in items):
        # we can concatenate along the first dimension
        return torch.cat(items)

    return items

Thanks for the issue! Mind try to reproduce with boring model and sharing the code?

Yes, I tried the boring model, and it worked fine. However with the same module but the different backbone, it crushed.
Firstly, I set gpus=1 and returned {'loss': loss} in training_step, and the error RuntimeError: grad can be implicitly created only for scalar outputs occured at first training step (Also I printed the returned item: {'loss': tensor(1755106.8750, device='cuda:0', grad_fn=<AddBackward0>)}). So I returned loss directly instead of the dictionary and it worked fine.
After that, I simply changed gpus to be 2, the error RuntimeError: All input tensors must be on the same device. Received cuda:1 and cuda:0 happened on epoch end.
I think there might be something wrong in collecting items and backwarding losses in training steps.

here is my module:
class ResNetVAE(pl.LightningModule):
    def __init__(
        self,
        lr,
        weight_decay,
        fc_hidden1=1024,
        fc_hidden2=1024,
        drop_p=0.2,
        CNN_embed_dim=256
    ):
        super().__init__()

        self.lr = lr
        self.weight_decay = weight_decay
        self.fc_hidden1, self.fc_hidden2, self.CNN_embed_dim = fc_hidden1, fc_hidden2, CNN_embed_dim

        # CNN architechtures
        self.ch1, self.ch2, self.ch3, self.ch4 = 16, 32, 64, 128
        self.k1, self.k2, self.k3, self.k4 = (5, 5), (3, 3), (3, 3), (3, 3)      # 2d kernal size
        self.s1, self.s2, self.s3, self.s4 = (2, 2), (2, 2), (2, 2), (2, 2)      # 2d strides
        self.pd1, self.pd2, self.pd3, self.pd4 = (0, 0), (0, 0), (0, 0), (0, 0)  # 2d padding

        # encoding components
        resnet = models.resnet152(pretrained=True)
        modules = list(resnet.children())[:-1]      # delete the last fc layer.
        self.resnet = nn.Sequential(*modules)
        self.fc1 = nn.Linear(resnet.fc.in_features, self.fc_hidden1)
        self.bn1 = nn.BatchNorm1d(self.fc_hidden1, momentum=0.01)
        self.fc2 = nn.Linear(self.fc_hidden1, self.fc_hidden2)
        self.bn2 = nn.BatchNorm1d(self.fc_hidden2, momentum=0.01)
        # Latent vectors mu and sigma
        self.fc3_mu = nn.Linear(self.fc_hidden2, self.CNN_embed_dim)      # output = CNN embedding latent variables
        self.fc3_logvar = nn.Linear(self.fc_hidden2, self.CNN_embed_dim)  # output = CNN embedding latent variables

        # Sampling vector
        self.fc4 = nn.Linear(self.CNN_embed_dim, self.fc_hidden2)
        self.fc_bn4 = nn.BatchNorm1d(self.fc_hidden2)
        self.fc5 = nn.Linear(self.fc_hidden2, 64 * 4 * 4)
        self.fc_bn5 = nn.BatchNorm1d(64 * 4 * 4)
        self.relu = nn.ReLU(inplace=True)

        # Decoder
        self.convTrans6 = nn.Sequential(
            nn.ConvTranspose2d(in_channels=64, out_channels=32, kernel_size=self.k4, stride=self.s4,
                               padding=self.pd4),
            nn.BatchNorm2d(32, momentum=0.01),
            nn.ReLU(inplace=True),
        )
        self.convTrans7 = nn.Sequential(
            nn.ConvTranspose2d(in_channels=32, out_channels=8, kernel_size=self.k3, stride=self.s3,
                               padding=self.pd3),
            nn.BatchNorm2d(8, momentum=0.01),
            nn.ReLU(inplace=True),
        )

        self.convTrans8 = nn.Sequential(
            nn.ConvTranspose2d(in_channels=8, out_channels=3, kernel_size=self.k2, stride=self.s2,
                               padding=self.pd2),
            nn.BatchNorm2d(3, momentum=0.01),
            nn.Sigmoid()
        )

    def loss_function(self, recon_x, x, mu, logvar):
        MSE = F.binary_cross_entropy(recon_x, x, reduction='sum')
        KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
        return MSE + KLD*weight_kld

    def training_step(self, batch, batch_idx):
        return self._step(batch, 'train_loss') # ['loss']

    def validation_step(self, batch, batch_idx):
        return self._step(batch, 'valid_loss')

    def _step(self, batch, name):
        x_reconst, z, mu, logvar = self._forward(batch)
        loss = self.loss_function(x_reconst, batch, mu, logvar)
        self.log(
            name,
            loss,
            on_step=True,
            on_epoch=True,
            prog_bar=True,
            logger=True,
        )
        return {'loss': loss}

    def configure_optimizers(self):
        return torch.optim.Adam(
            self.parameters(),
            lr=self.lr,
            weight_decay=self.weight_decay
        )

    def _encode(self, x):
        # ResNet
        x = self.resnet(x)
        x = x.view(x.size(0), -1)

        # FC layers
        x = self.bn1(self.fc1(x))
        x = self.relu(x)
        x = self.bn2(self.fc2(x))
        x = self.relu(x)
        mu, logvar = self.fc3_mu(x), self.fc3_logvar(x)
        return mu, logvar

    def _reparameterize(self, mu, logvar):
        std = logvar.mul(0.5).exp_()
        eps = std.data.new(std.size()).normal_()
        return eps.mul(std).add_(mu)

    def _decode(self, z):
        x = self.relu(self.fc_bn4(self.fc4(z)))
        x = self.relu(self.fc_bn5(self.fc5(x))).view(-1, 64, 4, 4)
        x = self.convTrans6(x)
        x = self.convTrans7(x)
        x = self.convTrans8(x)
        x = F.interpolate(x, size=(224, 224), mode='bilinear')
        return x

    def _forward(self, x):
        mu, logvar = self._encode(x)
        z = self._reparameterize(mu, logvar)
        x_reconst = self._decode(z)
        return x_reconst, z, mu, logvar

Hi @kchuang625!
I am trying to replicate it in colab (https://colab.research.google.com/drive/1nRhiaMFPdc8vh7hAX-u3bYGrpkGVowcb?usp=sharing) but i'm getting NameError: name 'weight_kld' is not defined
Can you update the snippet?

Hi @carmocca, thanks for reply!
Here is the updated notebook: https://colab.research.google.com/drive/1Ra1T6Jdqq8U0GNDrhZc37JgSzR8jYvFc?usp=sharing
Note.

input tensor should be in [0, 1] (for bce loss).

training and validating mode might have different behaviors, so I gave train and valid dataloader to trainer.

I'm getting the following error (not the reported error) when I try to run it on DP.
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 1024])
I'm assuming it's due to the model architecure. Can you take a look @kchuang625 ?

Oh! It's because there is batchNorm layer in the model. Simply modify the dataset and dataloader:
class RandomDataset(torch.utils.data.Dataset):
    def __getitem__(self, index):
        return torch.rand(3, 224, 224)

    def __len__(self):
        return 30

train_loader = torch.utils.data.DataLoader(
    RandomDataset(),
    batch_size=2,
)

valid_loader = torch.utils.data.DataLoader(
    RandomDataset(),
    batch_size=2,
)

Looks like this has been fixed already since it works with current master.
Note that you will have to update your step function to:
    def _step(self, batch, name):
        ...
        return loss  # was {'loss': loss}

Otherwise you'll get RuntimeError: grad can be implicitly created only for scalar outputs
Please try yourself and close this issue if it works for you 😄

@carmocca thanks for the reply!
I did test the script when pl 1.1.6 was released, and it turned out that changing the returned type from Dict to torch.Tensor is actually my temporary solution for now! It made me not need to manually modify the source code 😄
I have the impression that it's supported to return Dict with loss key in 0.x.x. (cuz it's really convenient to return other things by the way)
I wonder if this feature had been deprecated?

You are correct, it should work.
Do you mind opening a new issue about this? Since the original purpose of this one was to fix:
RuntimeError: All input tensors must be on the same device. Received cuda:1 and cuda:0
Tag me and i'll take a look. Thanks!


Measure clvm cost

These commits are best reviewed one at a time.
This PR:

adds a benchmark tool to aid in determining reasonable costs for the new BLS operators.
records the measurements in benchmarks.txt
updates the cost constants
updates the operator unit tests to reflect the new costs
updates the BLS test case generation script to reflect the new costs

As you can see from the benchmarks.txt, there's quite a lot of variance between runs, so the precision of these costs shouldn't be overestimated. I think as long as the cost is somewhat reasonable, we have quite a lot of wiggle room. I made some of the higher cost constants somewhat even numbers for instance.
Something else noteworthy is that when calibrating the timings and cost against point_add, the base-cost and the cost-per-arg were quite far apart. I used different scale factors for those. It's not clear to me why the previous way of measuring this would come up with different numbers, but it was quite a bit different machine (Intel Mac Laptop) compared to what I ran these one, AMD Ubuntu desktop.
I'm interested in any opinions on shifting the costs a bit. For example, richard suggested that g1_negate and g2_negate should be cheaper.

I'm interested in any opinions on shifting the costs a bit. For example, richard suggested that g1_negate and g2_negate should be cheaper.

This is based on a new implementation that just does bit-twiddling (and doesn't bother verifying for validity). Validity verification is done many other places, and if P is valid, the bit-twiddled version of P is also valid. My concern is that developers start doing the bit twiddling themselves with logxor to save cost.

yeah, it would be bad if people would start using xor instead of g1_negate and g2_negate, because that would prevent the future optimization of not converting everything to bytes eagerly.
On the other hand, it's hard to predict how expensive it would be to run with that optimization without implementing it. I'm a bit hesitant to not validate g1 points. We do that for point_add and I think it's a safety issue. in other words, it seems risky to widen the contract for g1_negate to accept invalid points.
That said, maybe we can just deduct some cost from it as a preparation for optimizing this, and disincentivizing circumventing the checks with xor.

@richardkiss cameron pointed out that the cost of AGG_SIG_* is a lot less than bls_verify. I'm thinking that maybe we can lower some costs in anticipation of future optimizations.
AGG_SIG_* is really the counterpart to cost-per-arg, since the block is one big aggregate signature. Even then, the cost of the operator is essentially 5 times as expensive.
I think it would be reasonable to cut all costs in half (except the per-byte).
Regarding the negate operators. I think the main risk with leaving them as expensive is that it creates an incentive to use xor. Using XOR will defeat the future optimization where we can skip the subgroup check, since we preserve the values. However, it's possible the negation on the native G1 and G2 point is still more expensive than the XOR.
I think the risk of actually implementing negation as XOR is that it creates a (possibly risky) loop-hole to get invalid points.
Do you have any concrete suggestions on what changes you think we should make to this PR?

I think we should add *_negate_unsafe and make it cheaper than logxor so there's no risk of people code-golfing in logxor. Then we can soft fork one out later if necessary once the concern about points outside the subgroup is better understood. I also understand if you don't want to do this due to time constraints.
I'm fine with reducing all costs by 50%. It's at worst just a mild DOS... "slow" blocks, which we'll likely be able to mitigate with optimizations.

@richardkiss @cameroncooper Please take a look at the most recent commit. I made these changes:

added test to establish cost for XORing 48 and 96 byte buffers, and made g1_negate and g2_negate match. here
implemented g1_negate as XOR, without validating the point. here.
implemented g2_negate as XOR without validating the point. here.
Updated most costs by cutting them in half (here), except:

per-byte-costs, which remained the same
the pairing per-arg costs. I made those match the cost of AGG_SIG_* conditions (1200000). Using the operators is still a bit more expensive since there's a base cost, and to verify a signature you need two arguments (message, pk) and (signature, generator). This is reasonable because in AGG_SIG_*, the second arg is amortized over all puzzles in the block.
the g1_negate and g2_negate match the cost of the corresponding XOR. Using these dedicated operators will still be cheaper than XOR because you don't need to include a large constant ot XOR with.


RollupError: "AUTH_SECRET" is not exported by "virtual:$env/static/private", imported by "src/hooks.server.ts"

Describe the bug
9: import {
10:   AUTH_SECRET,
^
11:   KEYCLOAK_ID,
12:   KEYCLOAK_SECRET,
error during build:
RollupError: "AUTH_SECRET" is not exported by "virtual:$env/static/private", imported by "src/hooks.server.ts".
at error (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/parseAst.js:337:30)
at Module.error (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:12738:16)
at Module.traceVariable (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:13175:29)
at ModuleScope.findVariable (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:11592:39)
at Identifier.bind (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:7179:40)
at Property.bind (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:4625:23)
at ObjectExpression.bind (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:4621:28)
at CallExpression.bind (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:4621:28)
at CallExpression.bind (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:8813:15)
at VariableDeclarator.bind (file:///drone/src/node_modules/vite/node_modules/rollup/dist/es/shared/node-entry.js:4625:23)
Reproduction
No response
Logs
No response
System Info
Need to install the following packages:
<EMAIL_ADDRESS>Ok to proceed? (y) y

  System:
    OS: macOS 14.2
    CPU: (10) arm64 Apple M2 Pro
    Memory: 120.92 MB / 32.00 GB
    Shell: 5.9 - /usr/local/bin/zsh
  Binaries:
    Node: 18.19.0 - ~/n/bin/node
    Yarn: 4.0.2 - ~/n/bin/yarn
    npm: 10.2.3 - ~/n/bin/npm
    bun: 1.0.18 - ~/.bun/bin/bun
  Browsers:
    Chrome: 120.0.6099.129
    Edge: 120.0.2210.89
    Safari: 17.2
  npmPackages:
    @sveltejs/adapter-auto: ^3.0.1 => 3.0.1
    @sveltejs/adapter-node: ^2.0.1 => 2.0.1
    @sveltejs/kit: ^2.0.4 => 2.0.4
    @sveltejs/vite-plugin-svelte: ^3.0.1 => 3.0.1
    svelte: ^4.2.8 => 4.2.8
    vite: ^5.0.10 => 5.0.10

Severity
blocking all usage of SvelteKit
Additional Information
"Dynamic environment variables can no longer be read during prerendering in SvelteKit 2 — you should use the static modules instead."
The Sveltekit 2 it changed the way we use dynamic private envs like import { env as privateEnv } from "$env/dynamic/private", so the recommendation is to use "$env/static/private".
What should we do now in a CI to be able to build?
My Drone CI build does not have any environment variables or static .env in the repository. The environment variables on my system are passed in the k8s deployment only during execution.

The only way now will be to include all environment variables in Drone CI so that sveltekit 2 can import via "$env/static/private" during the build.

Was this change in Sveltekit 2 really necessary? Would there be any other solution to work with private dynamic variables?
Now it seems that I am forced to include all my project variables in the CI build tool.

Are you using prerendering? (Trying to understand if this would be fixable by relaxing the constraint to only error when env variables are accessed during prerendering)

Hi @dummdidumm,
Looking now at the documentation, it says that prerendering is enabled by default.
This error happens in hooks.server.ts.
Should I disable prerendering on all layouts and/or pages? Is there a way to do this globally?

My src/routes/+layout.ts and
export const ssr = true;
export const prerender = false;

and still have problems with importing
Well, I'll have to downgrade to Sveltekit 1. It will be a hassle to have to include all the environment variables to be able to build the project.
Before, if I needed to change the value of a variable, I just had to redeploy the project in my Kubernetes, something that took seconds. Now I would have to do a new build to include a static variable, and it would go from seconds to at least 10 minutes to do a build.

Ok so you're not using prerendering - in that case, it should be possible to keep using $env/dynamic/.. and we need to fix it

Ok so you're not using prerendering - in that case, it should be possible to keep using $env/dynamic/.. and we need to fix it

Thank you very much @dummdidumm!!

Thanks!

thanks!

thanks!

It seems to be still not working for me, using the same prerender = false I am getting during vite build:
"DISK_ROOT" is not exported by "virtual:$env/dynamic/private", imported by "src/routes/trim/+server.js".
file: C:/ ... /src/routes/trim/+server.js:1:9
1: import { DISK_ROOT, OUT_DIR, SERVER_PATH } from '$env/dynamic/private';
            ^
2:
3: import { json } from '@sveltejs/kit';
error during build:
RollupError: "DISK_ROOT" is not exported by "virtual:$env/dynamic/private", imported by "src/routes/trim/+server.js".
    at error (file:///C:/ ... /node_modules/.pnpm/rollup@4.9.5/node_modules/rollup/dist/es/shared/parseAst.js:337:30)
    at Module.error (file:///C:/ ... /node_modules/.pnpm/rollup@4.9.5/node_modules/rollup/dist/es/shared/node-entry.js:12759:16)

During vite dev no problem.
"@sveltejs/kit": "^2.0.0"

It seems to be still not working for me, using the same prerender = false I am getting during vite build:
"DISK_ROOT" is not exported by "virtual:$env/dynamic/private", imported by "src/routes/trim/+server.js".
file: C:/ ... /src/routes/trim/+server.js:1:9
1: import { DISK_ROOT, OUT_DIR, SERVER_PATH } from '$env/dynamic/private';
            ^
2:
3: import { json } from '@sveltejs/kit';
error during build:
RollupError: "DISK_ROOT" is not exported by "virtual:$env/dynamic/private", imported by "src/routes/trim/+server.js".
    at error (file:///C:/ ... /node_modules/.pnpm/rollup@4.9.5/node_modules/rollup/dist/es/shared/parseAst.js:337:30)
    at Module.error (file:///C:/ ... /node_modules/.pnpm/rollup@4.9.5/node_modules/rollup/dist/es/shared/node-entry.js:12759:16)

During vite dev no problem.
"@sveltejs/kit": "^2.0.0"

Update to the latest version of 2.0 which should resolve the issue.

I have just tried, "@sveltejs/kit": "^2.3.2", still the same, unfortunately.

Actually, if this helps, I am trying to import .env variables in API routes.

Try running svelte-kit sync or remove .svelte-kit completely.

I am afraid this does not work either. OK, what I did is I created the fresh 'skeleton' project, JS + JSDoc:
pnpm create svelte@latest my-app
cd my-app
pnpm install
pnpm run dev

pnpm install output:
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: C:\Users\...\AppData\Local\pnpm\store\v3
  Virtual store is at:             node_modules/.pnpm
Progress: resolved 285, reused 250, downloaded 0, added 250, done

devDependencies:
+ @sveltejs/adapter-auto 3.1.0
+ @sveltejs/kit 2.3.2
+ @sveltejs/vite-plugin-svelte 3.0.1
+ @types/eslint 8.56.0 (8.56.2 is available)
+ eslint 8.56.0
+ eslint-config-prettier 9.1.0
+ eslint-plugin-svelte 2.35.1
+ prettier 3.2.2
+ prettier-plugin-svelte 3.1.2
+ svelte 4.2.8
+ svelte-check 3.6.3
+ typescript 5.3.3
+ vite 5.0.11
+ vitest 1.2.0

Done in 8.6s

I added a /src/routes/+layout.js with:
export const prerender = false;

I added /src/routes/+server.js with:
import { EXAMPLE } from "$env/dynamic/private"

Created also a .env file with:
EXAMPLE=test

Next, issued pnpm run build and the result was:
><EMAIL_ADDRESS>build C:\Users\...\Documents\js\my-app
> vite build

vite v5.0.11 building SSR bundle for production...
src/routes/+server.js (1:9) "EXAMPLE" is not exported by "virtual:$env/dynamic/private", imported by "src/routes/+server.js".
✓ 88 modules transformed.
vite v5.0.11 building for production...
✓ 60 modules transformed.
.svelte-kit/output/client/_app/version.json                             0.03 kB │ gzip:  0.05 kB
.svelte-kit/output/client/.vite/manifest.json                           2.26 kB │ gzip:  0.46 kB
.svelte-kit/output/client/_app/immutable/entry/start.IRGdKFW6.js        0.07 kB │ gzip:  0.08 kB
.svelte-kit/output/client/_app/immutable/nodes/0._8CBVaDm.js            0.60 kB │ gzip:  0.39 kB
.svelte-kit/output/client/_app/immutable/nodes/2.HmgQHYm4.js            0.69 kB │ gzip:  0.45 kB
.svelte-kit/output/client/_app/immutable/nodes/1.DWgraoQ-.js            0.97 kB │ gzip:  0.57 kB
.svelte-kit/output/client/_app/immutable/chunks/scheduler.zMJaRgub.js   2.16 kB │ gzip:  1.02 kB
.svelte-kit/output/client/_app/immutable/chunks/index.yy3khgz-.js       5.25 kB │ gzip:  2.21 kB
.svelte-kit/output/client/_app/immutable/entry/app.H6g-Nt4r.js          5.96 kB │ gzip:  2.41 kB
.svelte-kit/output/client/_app/immutable/chunks/entry.yQqEGdCS.js      25.76 kB │ gzip: 10.35 kB
✓ built in 749ms
.svelte-kit/output/server/.vite/manifest.json                   2.12 kB
.svelte-kit/output/server/entries/endpoints/_server.js          0.07 kB
.svelte-kit/output/server/entries/fallbacks/layout.svelte.js    0.24 kB
.svelte-kit/output/server/internal.js                           0.25 kB
.svelte-kit/output/server/entries/pages/_page.svelte.js         0.37 kB
.svelte-kit/output/server/entries/fallbacks/error.svelte.js     1.16 kB
.svelte-kit/output/server/chunks/ssr.js                         3.35 kB
.svelte-kit/output/server/chunks/internal.js                    5.90 kB
.svelte-kit/output/server/chunks/exports.js                     5.96 kB
.svelte-kit/output/server/index.js                            113.99 kB

Run npm run preview to preview your production build locally.

> Using @sveltejs/adapter-auto
  Could not detect a supported production environment. See https://kit.svelte.dev/docs/adapters to learn how to configure your app to run on the platform of your choosing
  ✔ done
✓ built in 2.42s
 *  Terminal will be reused by tasks, press any key to close it.

Doesn't work for me — neither with the new skeleton project nor with the old one updated to 2.4.3.
error during build: RollupError: "MONGO_URL" is not exported by "virtual:$env/dynamic/private", imported by "src/db/connection.js".
Please, help. Am I missing something?

Having the exact same issue - having to resort to using the VITE_ prefix with a process.env call
	"devDependencies": {
		"@sveltejs/adapter-auto": "^3.0.0",
		"@sveltejs/kit": "^2.0.0",
		"@sveltejs/vite-plugin-svelte": "^3.0.0",
		"svelte": "^4.0.5",
		"vite": "^5.0.0"
	},

Found the solution and expectedly feel stupid, I guess I didn't read the documentation properly.  Based on the error messages, it looks like it's the same issue both @janguardian and @PlkMarudny ran into.
In Sveltekit, dynamic and static environment variables are accessed differently.
Static
import { API_KEY } from '$env/static/private';

console.log(API_KEY)

Dynamic
import { env } from '$env/dynamic/private

console.log(env.API_KEY)

Doesn't work for me — neither with the new skeleton project nor with the old one updated to 2.4.3.
error during build: RollupError: "MONGO_URL" is not exported by "virtual:$env/dynamic/private", imported by "src/db/connection.js".
Please, help. Am I missing something?

Check your .gitignore file,maybe .env in the ignore list

I am afraid this does not work either. OK, what I did is I created the fresh 'skeleton' project, JS + JSDoc:
pnpm create svelte@latest my-app
cd my-app
pnpm install
pnpm run dev

pnpm install output:
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: C:\Users\...\AppData\Local\pnpm\store\v3
  Virtual store is at:             node_modules/.pnpm
Progress: resolved 285, reused 250, downloaded 0, added 250, done

devDependencies:
+ @sveltejs/adapter-auto 3.1.0
+ @sveltejs/kit 2.3.2
+ @sveltejs/vite-plugin-svelte 3.0.1
+ @types/eslint 8.56.0 (8.56.2 is available)
+ eslint 8.56.0
+ eslint-config-prettier 9.1.0
+ eslint-plugin-svelte 2.35.1
+ prettier 3.2.2
+ prettier-plugin-svelte 3.1.2
+ svelte 4.2.8
+ svelte-check 3.6.3
+ typescript 5.3.3
+ vite 5.0.11
+ vitest 1.2.0

Done in 8.6s

I added a /src/routes/+layout.js with:
export const prerender = false;

I added /src/routes/+server.js with:
import { EXAMPLE } from "$env/dynamic/private"

Created also a .env file with:
EXAMPLE=test

Next, issued pnpm run build and the result was:
><EMAIL_ADDRESS>build C:\Users\...\Documents\js\my-app
> vite build

vite v5.0.11 building SSR bundle for production...
src/routes/+server.js (1:9) "EXAMPLE" is not exported by "virtual:$env/dynamic/private", imported by "src/routes/+server.js".
✓ 88 modules transformed.
vite v5.0.11 building for production...
✓ 60 modules transformed.
.svelte-kit/output/client/_app/version.json                             0.03 kB │ gzip:  0.05 kB
.svelte-kit/output/client/.vite/manifest.json                           2.26 kB │ gzip:  0.46 kB
.svelte-kit/output/client/_app/immutable/entry/start.IRGdKFW6.js        0.07 kB │ gzip:  0.08 kB
.svelte-kit/output/client/_app/immutable/nodes/0._8CBVaDm.js            0.60 kB │ gzip:  0.39 kB
.svelte-kit/output/client/_app/immutable/nodes/2.HmgQHYm4.js            0.69 kB │ gzip:  0.45 kB
.svelte-kit/output/client/_app/immutable/nodes/1.DWgraoQ-.js            0.97 kB │ gzip:  0.57 kB
.svelte-kit/output/client/_app/immutable/chunks/scheduler.zMJaRgub.js   2.16 kB │ gzip:  1.02 kB
.svelte-kit/output/client/_app/immutable/chunks/index.yy3khgz-.js       5.25 kB │ gzip:  2.21 kB
.svelte-kit/output/client/_app/immutable/entry/app.H6g-Nt4r.js          5.96 kB │ gzip:  2.41 kB
.svelte-kit/output/client/_app/immutable/chunks/entry.yQqEGdCS.js      25.76 kB │ gzip: 10.35 kB
✓ built in 749ms
.svelte-kit/output/server/.vite/manifest.json                   2.12 kB
.svelte-kit/output/server/entries/endpoints/_server.js          0.07 kB
.svelte-kit/output/server/entries/fallbacks/layout.svelte.js    0.24 kB
.svelte-kit/output/server/internal.js                           0.25 kB
.svelte-kit/output/server/entries/pages/_page.svelte.js         0.37 kB
.svelte-kit/output/server/entries/fallbacks/error.svelte.js     1.16 kB
.svelte-kit/output/server/chunks/ssr.js                         3.35 kB
.svelte-kit/output/server/chunks/internal.js                    5.90 kB
.svelte-kit/output/server/chunks/exports.js                     5.96 kB
.svelte-kit/output/server/index.js                            113.99 kB

Run npm run preview to preview your production build locally.

> Using @sveltejs/adapter-auto
  Could not detect a supported production environment. See https://kit.svelte.dev/docs/adapters to learn how to configure your app to run on the platform of your choosing
  ✔ done
✓ built in 2.42s
 *  Terminal will be reused by tasks, press any key to close it.

Check your .gitignore file,maybe .env in the ignore list


npm build inside docker build does not pack static media file

See Build Docker Image #30

Remove git lfs works


Integrate Auto-Centering Code into Drive Class (or it's own class)

Auto-Centering code is currently commented out and still inside the main while loop

completed


Window not responding

I tried the noise example by building from source. Unfortunately, it takes up 30-35% of CPU usage and the window immediately is not responding. This happens as soon as I (and even if I only) invoke Window::new.
I have a Dell Inspiron laptop with Intel i7 16gb, Windows 10, Intel GPU.
Is this expected? I'm trying to use this for stdg but if the window doesn't respond I can't.

That sounds strange. Have you tried to just do something like this
 while window.is_open() && !window.is_key_down(Key::Escape) {
        window.get_keys().map(|keys| {
            for t in keys {
                match t {
                    Key::W => println!("holding w!"),
                    Key::T => println!("holding t!"),
                    _ => (),
                }
            }
        });

        window.update_with_buffer(&buffer).unwrap();
    }

And see how that works? Also regarding CPU time. The code currently will try to run as fast as it can. It can be a good idea to limit the frame rate (add sleeping) so it doesn't try to consume all the CPU time.

Hi @emoon. Sorry for not closing this earlier.
This isn't actually an issue at all with MiniFB at all. The problem was that I was not calling window.update_with_buffer(&buffer).unwrap(); as often as possible in an infinite loop, which seems to be necessary. Everything seems to be working smoothly now that I do that.
Thank you for creating this library, btw. I'm finding it very useful at the moment.

Cool. np :)


Add support for Enchantment Descriptions.

This PR adds support for Enchantment Descriptions and other mods with similar effects.

Awesome! We allays appreciate contributions thanks for your work.
This will be included in the next update.
:)


pytest plugin

I suggest a pytest plugin to allow the following.
pip install pytest-pyhttptest

The above installs the pytest plugin and the httptest fixture.
def test_get(httptest):
    httptest.run(either a json file path, or a json string)

Why?
The advantege is that this goes together with pytest reports

Hello @rochacbruno,
I think we can do that, but I don't want to mix it up with this package. Because pyhttptest purpose is only one just a command-line tool. The idea is very good and I totally agree with you. We can do it on a separate project!
Once again thank you!
Contact me for more about pytest-pyhttptest if you want.


list_phones.py should go in src, not phones?

Proposal; move list_phones.py to its parent directory so we don't have a mix of code and data in the same directory. @ajmalanoski @jacksonllee thoughts?

I agree with the proposal. I think it makes sense to keep the code and data separate.


Splat route

While developing my web app, it's handy to have the server to serve up the site regardless of the route. All my routing is being handled on the front end.

Same as #148 and #80

Thanks @jantimon. Duplicate of #80.


Breaks Session Manager completely

Hi -installed this plugin and found that Session Manager no longer saves sessions correctly. Automatic session backups result in saved sessions with many thousands of (non-existing) tabs which cannot be restored. Even the sessions saved before installing this plugin cannot be loaded any more - resulting error is:
This operation failed due to a file access error:
window.tabs is undefined(TypeError)
half way restored...-2017-03-13.session
SessionCookiesInternal.getHostsForWindow@resource://app/modules/sessionstore/SessionCookies.jsm:105:1
SessionCookiesInternal.update@resource://app/modules/sessionstore/SessionCookies.jsm:65:19
this.SessionCookies<.update@resource://app/modules/sessionstore/SessionCookies.jsm:28:5
SessionStoreInternal.getCurrentState@resource:///modules/sessionstore/SessionStore.jsm:2762:5
ssi_getBrowserState@resource:///modules/sessionstore/SessionStore.jsm:2001:17
ss_getBrowserState@resource:///modules/sessionstore/SessionStore.jsm:217:12
Private.getSessionState@chrome://sessionmanager/content/modules/session_data_processing.jsm:61:81
this.SessionDataProcessing.getSessionState@chrome://sessionmanager/content/modules/session_data_processing.jsm:32:10
... etc etc ...
Disabling this plugin did not seem to help - had to remove it completely.
Really loved the plugin, but unfortunately have to choose, and session backup is of higher priority for me.
Thanks,
Andrej

I would recommend creating a new profile and installing both Copy Urls
Expert and Session Manager in that profile. Then restore the session. Make
sure that there is no other extension installed in that new profile.
Copy Urls Expert does not modify Firefox default behavior in any way.  I
have received complaints in the past of issues similar to this but every
time there is some another extension causing incompatability. I can confirm
that I use both CUE and Session Manager together and never had problems in
restoring sessions.
On Mar 13, 2017 11:43 AM, "afalout"<EMAIL_ADDRESS>wrote:

Hi -installed this plugin and found that Session Manager no longer saves
sessions correctly. Automatic session backups result in saved sessions with
many thousands of (non-existing) tabs which cannot be restored. Even the
sessions saved before installing this plugin cannot be loaded any more -
resulting error is:
This operation failed due to a file access error:
window.tabs is undefined(TypeError)
half way restored...-2017-03-13.session
SessionCookiesInternal.getHostsForWindow@resource://
app/modules/sessionstore/SessionCookies.jsm:105:1
SessionCookiesInternal.update@resource://app/modules/
sessionstore/SessionCookies.jsm:65:19
this.SessionCookies<.update@resource://app/modules/
sessionstore/SessionCookies.jsm:28:5
SessionStoreInternal.getCurrentState@resource:///modules/sessionstore/
SessionStore.jsm:2762:5
ssi_getBrowserState@resource:///modules/sessionstore/
SessionStore.jsm:2001:17
ss_getBrowserState@resource:///modules/sessionstore/
SessionStore.jsm:217:12
Private.getSessionState@chrome://sessionmanager/
content/modules/session_data_processing.jsm:61:81
this.SessionDataProcessing.getSessionState@chrome://
sessionmanager/content/modules/session_data_processing.jsm:32:10
... etc etc ...
Disabling this plugin did not seem to help - had to remove it completely.
Really loved the plugin, but unfortunately have to choose, and session
backup is of higher priority for me.
Thanks,
Andrej
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kashiif/copy-urls-expert/issues/41, or mute the
thread
https://github.com/notifications/unsubscribe-auth/ABveTHJhrKVZkKjCtM8D2nf0GoL8Qp1Pks5rlOWqgaJpZM4Ma5ch
.

Hi Kashif, thanks for your reply; Having CUE work in a separate profile without all the plugins I need wont help me - even if it worked. Here are some details because of which I believe that CUE is in some way involved in this:
1 - I have Firefox installed on 11 machines, including 2 Linux machines , 2 Android tablets and Windows
2 - All have sync activated, so exactly the same plugins are installed, with exactly the same settings propagated across all of them
3 - I did not change any Firefox settings, plugins, or plugins settings for at least 5 months. All machines and all Firefox installations in that time did not have absolutely any issues.
4 - After installing CUE on >ONE< Linux machine, that instance of Firefox started crashing every few hours (that is how I discovered that Session Manager "Restore Session" does not work any more). As none of the other Firefox instances where restarted in that time, CUE was downloaded by sync - but not installed/activated - so they continued to work fine.
5 - When I removed CUE from that one Linux machine, after 3 days of crashing and SM issues, Firefox crashing stopped immediately, and Session Manager started working again.
6 - Since that time 4 days passed without a single Firefox crash.
Now I do appreciate that you believe that CUE does not modify Firefox behaviour in any way, but I would say that something, somewhere, somehow related to CUE does - directly of indirectly - does.
I often have large number of windows open in FF (10-15) and on average 200 tabs open at any time. If that matters. I noticed that sessions saved that would not load show between 3000 and 5000 tabs on them. Trying to open ANY saved sessions while CUE was installed would restore all windows, but only first window will have tabs restored.
Not sure this is of any help to you, but I hope you will keep it in mind and possibly one day spot something that might cause described behaviour.
All the best,
Andrej

@afalout I also use both addons. No issues for me eigther. Like the dev said, try to install both addons on a new clean/fresh firefox profile and try to load/restore your 2017-03-13.session problematic session.

Not enough information on this issue. I use both CUE and SessionManager with Firefox and Palemoon on Windows and Mac machines without any issue. Also see above comment.
Closing, feel free to open if it is reproducible.


convert accounts and keys #139 #140

Please, write some description on this scripts: how to run, which components need install before run.
Please check, that this scripts works correct:

compile and install cyberway
run cyberway and create some users with some authority
add data from golos with this scripts
check that cyberway correct works with new data

depend:
pymongo
run:
convert.py

It's better to place some README file with description about purpose of scripts into directory with scripts.


Add Projects URLs

@master-caster you probably should rebase your branch from upstream in order to avoid the conflict.


preserve non-JSON lines during normalization

Fixes https://github.com/laumann/compiletest-rs/issues/169
I am not sure what the usual process is, do you want the files here to be verbatim identical to what is in rustc? The diff seems quite large currently, at least for runtest.rs. I will submit the same thing there as well, if it still applies.

@Manishearth @laumann The corresponding rustc PR has been r+'d.  Does that mean we can land this here?
Both Miri and Clippy had missed test failures because of this, so it seems prudent to fix this quickly.
Ideally we'd also carry over the part from rustc's compiletest that checks the exit code of the compiler. However, I have no idea how upstream syncing is handled -- see https://github.com/laumann/compiletest-rs/issues/175.

Sure, feel free to upstream
On Wed, Apr 10, 2019, 7:45 AM Ralf Jung<EMAIL_ADDRESS>wrote:

@Manishearth https://github.com/Manishearth @laumann
https://github.com/laumann The corresponding rustc PR
https://github.com/rust-lang/rust/pull/59769 has been r+'d. Does that
mean we can land this here?
Both Miri and Clippy had missed test failures because of this, so it seems
prudent to fix this quickly.
Ideally we'd also carry over the part from rustc's compiletest that checks
the exit code of the compiler. However, I have no idea how upstream syncing
is handled -- see #175
https://github.com/laumann/compiletest-rs/issues/175.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/laumann/compiletest-rs/pull/172#issuecomment-481721382,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABivSHWNER02ACxGcrLgZmd3L2VzcQlTks5vffkWgaJpZM4cgpCL
.

WDYM? I upstreamed this, it is waiting-for-bors. I am asking for it to also land here.

That's what I meant by "upstream", should have said "uplift", sorry.
To be clear: go ahead and make a version of your PR for this repo

Uh... this very PR we are talking in is a version of that PR for this repo...?

Oh, I thought this was an issue, my bad
(I don't have great internet right now)
On Thu, Apr 11, 2019, 1:51 AM Ralf Jung<EMAIL_ADDRESS>wrote:

Uh... this very PR we are talking in is a version of that PR for this
repo...?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/laumann/compiletest-rs/pull/172#issuecomment-482027657,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABivSJQZldGlQ0ZvbAr9jEsHYrXGfI--ks5vfveSgaJpZM4cgpCL
.

That's okay. :)  I was just terribly confused. ;)

The upstream PR has been merged, it would be nice to get this into compiletest-rs, too =)

(Note that CI is red for different reasons that are fixed by https://github.com/laumann/compiletest-rs/pull/170.)

Rebased to rerun CI.

CI has a happy green :)

@Manishearth can we please merge this? I just spent 2h debugging a really weird issue in Miri where CI on Windows seemed to not catch a problem, until realizing that actually it was this ICEs-are-ignored issue again.


fix(checkout): CHECKOUT-4936 Display shipping step with custom item c…

…arts
What?
Displaying the shipping step if a cart only has custom items.
Why?
Custom items are pseudo-physical items per our existing functionality in manual orders and LCO.
Currently draft orders with only custom items can't adjust a shipping address adjusted in checkout, and S2S carts consisting of only custom items cannot complete a checkout due to a missing shipping method error.
By surfacing the shipping step when custom items are present, draft orders can have their shipping address adjusted by shoppers, and checkout can be completed when carts created via S2S API only have custom items.
Testing / Proof
Before:

After:

@bigcommerce/checkout

@bigcommerce/cpoe-pod Can this be merged or is there something else that has to be done first?


executor: [ISSUE#481] Add syscall syz_open_files_in_dir

Allow opening all files in a dir for procfs/sysfs.
Example:
syz_open_files_in_dir$sys_power(file ptr[in, string["/sys/power"]], flag
s flags[open_flags], mode const[0]) fd_sys

Before sending a pull request, please review Contribution Guidelines:
https://github.com/google/syzkaller/blob/master/docs/contributing.md

I don't completely follow the intention of this function.
syz_open_files_in_dir claims to return an fd in the description, but it actually never returns an fd, it always returns 0. It does not have any randomness (besides flags and mode which are not very interesting), nor help fuzzer in any way. What this function does can be tested by running cat /sys/power/* once.
I think such function should open one of the files and return its fd to the fuzzer, so that it can call read/write/etc on the fd.
It would be reasonable to express this on the description level as follows:
openat$sys(fd const[AT_FDCWD], file ptr[in, string["/sys/*"]], flags flags[open_flags], mode const[0]) fd

and then in host.DetectSupportedSyscalls replace the file name with all present files that satisfy the mask.
Open questions:

Do we want to scan all files recursively, or not? Do we want to support something like /sys/** or /sys/**/* for recursive case?
host.DetectSupportedSyscalls is only called by the first fuzzer, subsequent fuzzer runs reuse the list of enabled syscalls (for performance reasons). How do we share/pass the file list between fuzzers?

issue #481

Good question for the fd, I didn't realize what those fd used for previously.
Now we are openating files under /sys/** or /sys/**/*, and then pass the fd to write/read. Is it making sense to have such functions?
But if I need to pass the fds to write/read, how it can be implemented since there are many fds?
And for question 2, we want to write random data into these sys nodes, so it's not just first run.

Now we are openating files under /sys/** or /sys/**/*, and then pass the fd to write/read. Is it making sense to have such functions?

I am not sure I fully understand the question. If by "now" you mean your pull request, then, no, it is not passing any fd's to read/write. Yes, it makes sense to pass fd's to read/write.

But if I need to pass the fds to write/read, how it can be implemented since there are many fds?

As I said, the function need to open just 1 file and return its fd.

ok, it's yes path. How's syscall like this, write_r which open each files under sys and then write to the fd. The purpose is to use random data syzkaller provided to these sys node. If ok to have such loop write, then we can have read_r too.
Another thought, can we have a for loop in descriptor file then we can include open, write inside the loop?

syzkaller can already read and write to file descriptors. No need to invent new, complex and excessive mechanisms. We just need to give it an fd.

I know that, but the sys node exist on one device might not available for another devices. We have plenty of devices to support. The current way we are doing now is openat, write and read. But if there is a complicated for loop syscall, we just need to enum online nodes and write.

There are ~40k sysfs nodes, in current way, it is painful to write the descriptor files.

package host is named somewhat confusingly, it runs on the target machine. So DetectSupportedSyscalls could just read contents of /sys.


Request - fromiso/frugal to full install

A request.
At the moment, if user boots fromiso or is running a frugal install on lets say sda4, it is not possible to do a normal install onto any other partition eg sda3. The cli-installer can do this, but the gui one doesn't show any options. Is it possible to change this?

at the moment you need to adjust the installer.conf file to not apply the
boot device filter.
I don't think there is a commandline switch but will check.
On Fri, May 1, 2020 at 11:54 AM anticapitalista<EMAIL_ADDRESS>wrote:

A request.
At the moment, if user boots fromiso or is running a frugal install on
lets say sda4, it is not possible to do a normal install onto any other
partition eg sda3. The cli-installer can do this, but the gui one doesn't
show any options. Is it possible to change this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/gazelle-installer/gazelle-installer/issues/113, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTAQ7ML7BNKUX4GBS7FIJ3RPLWBVANCNFSM4MXF2QTQ
.

there is not a command line switch at the moment.  the value to change in
/usr/share/gazelle-installer-data/installer.conf is
"INSTALL_FROM_ROOT_DEVICE".
On Fri, May 1, 2020 at 12:05 PM Dolphin Oracle<EMAIL_ADDRESS>wrote:

at the moment you need to adjust the installer.conf file to not apply the
boot device filter.
I don't think there is a commandline switch but will check.
On Fri, May 1, 2020 at 11:54 AM anticapitalista<EMAIL_ADDRESS>wrote:

A request.
At the moment, if user boots fromiso or is running a frugal install on
lets say sda4, it is not possible to do a normal install onto any other
partition eg sda3. The cli-installer can do this, but the gui one doesn't
show any options. Is it possible to change this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/gazelle-installer/gazelle-installer/issues/113, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTAQ7ML7BNKUX4GBS7FIJ3RPLWBVANCNFSM4MXF2QTQ
.

I completely missed that one. Tested installing a frugal install to a full install after editing that file. Worked a treat.


Implementation of Unsupervised RNN language model + trained embeddings

Hi,
I can't find the implementation of this model that was added recently to the results, and that appears to perform much better than the other ones.

I put that model in the seq2seq folder. It was kind of an experimental model where I generated answers for each question using an RNN model like Karpathy's char-rnn, then used the generated answers instead of the questions for the Embedding model (this might be cheating). But once you generate the answers for all the questions you can change this line.


IntegrityError at /vre-api-test/api/workflows/

When we start VRE and open the API from the browser at :vre-api-test/api/workflows/ we get the following error:
Environment:

Request Method: GET
Request URL: http://localhost:8000/vre-api-test/api/workflows/

Django Version: 4.0.10
Python Version: 3.11.3
Installed Applications:
['cells',
 'workflows',
 'virtual_labs',
 'data_products',
 'rest_framework',
 'rest_framework_gis',
 'corsheaders',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.gis']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django-keycloak-auth.middleware.KeycloakMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback (most recent call last):
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)

The above exception (update or delete on table "workflows_workflow" violates foreign key constraint "data_products_datapr_workflow_id_4d36ce10_fk_workflows" on table "data_products_dataproduct"
DETAIL:  Key (argo_id)=(n-a-a-vre-dev-skoulouzis-6zvkb) is still referenced from table "data_products_dataproduct".
) was the direct cause of the following exception:
  File "/opt/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/opt/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/venv/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/venv/lib/python3.11/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/venv/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/opt/venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/opt/venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/opt/venv/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/app/workflows/views.py", line 98, in list
    wf_serializer.save()
  File "/app/workflows/serializers.py", line 10, in save
    return super().save(**kwargs)
  File "/opt/venv/lib/python3.11/site-packages/rest_framework/serializers.py", line 726, in save
    self.instance = self.update(self.instance, validated_data)
  File "/app/workflows/serializers.py", line 29, in update
    workflow.delete()
  File "/opt/venv/lib/python3.11/site-packages/django/db/models/base.py", line 1094, in delete
    return collector.delete()
  File "/opt/venv/lib/python3.11/site-packages/django/db/models/deletion.py", line 446, in delete
    count = sql.DeleteQuery(model).delete_batch(
  File "/opt/venv/lib/python3.11/site-packages/django/db/models/sql/subqueries.py", line 42, in delete_batch
    num_deleted += self.do_query(
  File "/opt/venv/lib/python3.11/site-packages/django/db/models/sql/subqueries.py", line 20, in do_query
    cursor = self.get_compiler(using).execute_sql(CURSOR)
  File "/opt/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1361, in execute_sql
    cursor.execute(sql, params)
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/opt/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)

Exception Type: IntegrityError at /vre-api-test/api/workflows/
Exception Value: update or delete on table "workflows_workflow" violates foreign key constraint "data_products_datapr_workflow_id_4d36ce10_fk_workflows" on table "data_products_dataproduct"
DETAIL:  Key (argo_id)=(n-a-a-vre-dev-skoulouzis-6zvkb) is still referenced from table "data_products_dataproduct".

We manually triggered an update in Tilt for the DB and API and get the same error

Fixed by https://github.com/QCDIS/VREPaaS/issues/159


flink cdc pg 不支持中文表名和中文列

Search before asking

[X] I searched in the issues and found nothing similar.

Flink version
1.70
Flink CDC version
2.4.1
Database and its version
pg 13.5
Minimal reproduce step
flink cdc 连接pg
What did you expect to see?
希望同步pg 中文列或者中文表名不乱码。
What did you see instead?

同步带中文列的字段或者是中文表时，接收到的日志显示列是乱码。
Anything else?
No response
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

Considering collaboration with developers around the world, please re-create your issue in English on Apache Jira under project Flink with component tag Flink CDC. Thank you!


[I][BLE_scan.ino:30] my_gap_event_handler(): custom gap event handler, event: 3
Devices found: 3
Scan done!

Again, the ESP32 only sees the device when scanning with a app. I'll try to better understand, because using the BLE_client.ino example with increased scan intervalls (2000/1500 finds the device.

If you don't use WiFi try with interval equal window.

Can you please give me a hint on how to do "interval equal window"?

Sketch BLE_client.ino
Example sketch with changed pBLEScan->setInterval(2000); pBLEScan->setWindow(1500); and adjusted to my UUIDs
static BLEUUID serviceUUID("00001814-0000-1000-8000-00805f9b34fb");
static BLEUUID    charUUID("0000ff00-0000-1000-8000-00805f9b34fb");

/**
 * A BLE client example that is rich in capabilities.
 * There is a lot new capabilities implemented.
 * author unknown
 * updated by chegewara
 */

#include "BLEDevice.h"
//#include "BLEScan.h"

// The remote service we wish to connect to.
static BLEUUID serviceUUID("00001814-0000-1000-8000-00805f9b34fb");
// The characteristic of the remote service we are interested in.
static BLEUUID    charUUID("0000ff00-0000-1000-8000-00805f9b34fb");

static boolean doConnect = false;
static boolean connected = false;
static boolean doScan = false;
static BLERemoteCharacteristic* pRemoteCharacteristic;
static BLEAdvertisedDevice* myDevice;

static void notifyCallback(
  BLERemoteCharacteristic* pBLERemoteCharacteristic,
  uint8_t* pData,
  size_t length,
  bool isNotify) {
    Serial.print("Notify callback for characteristic ");
    Serial.print(pBLERemoteCharacteristic->getUUID().toString().c_str());
    Serial.print(" of data length ");
    Serial.println(length);
    Serial.print("data: ");
    Serial.println((char*)pData);
}

class MyClientCallback : public BLEClientCallbacks {
  void onConnect(BLEClient* pclient) {
  }

  void onDisconnect(BLEClient* pclient) {
    connected = false;
    Serial.println("onDisconnect");
  }
};

static void my_gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t* param) {
	ESP_LOGW(LOG_TAG, "custom gattc event handler, event: %d", (uint8_t)event);
        if(event == ESP_GATTC_DISCONNECT_EVT) {
                Serial.print("Disconnect reason: ");
                Serial.println((int)param->disconnect.reason);
        }
}

static void my_gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gatts_cb_param_t* param) {
	ESP_LOGI(LOG_TAG, "custom gatts event handler, event: %d", (uint8_t)event);
}

static void my_gap_event_handler(esp_gap_ble_cb_event_t  event, esp_ble_gap_cb_param_t* param) {
	ESP_LOGI(LOG_TAG, "custom gap event handler, event: %d", (uint8_t)event);
}

bool connectToServer() {
    Serial.print("Forming a connection to ");
    Serial.println(myDevice->getAddress().toString().c_str());

    BLEClient*  pClient  = BLEDevice::createClient();
    Serial.println(" - Created client");

    pClient->setClientCallbacks(new MyClientCallback());

    // Connect to the remove BLE Server.
    pClient->connect(myDevice);  // if you pass BLEAdvertisedDevice instead of address, it will be recognized type of peer device address (public or private)
    Serial.println(" - Connected to server");

    // Obtain a reference to the service we are after in the remote BLE server.
    BLERemoteService* pRemoteService = pClient->getService(serviceUUID);
    if (pRemoteService == nullptr) {
      Serial.print("Failed to find our service UUID: ");
      Serial.println(serviceUUID.toString().c_str());
      pClient->disconnect();
      return false;
    }
    Serial.println(" - Found our service");

    // Obtain a reference to the characteristic in the service of the remote BLE server.
    pRemoteCharacteristic = pRemoteService->getCharacteristic(charUUID);
    if (pRemoteCharacteristic == nullptr) {
      Serial.print("Failed to find our characteristic UUID: ");
      Serial.println(charUUID.toString().c_str());
      pClient->disconnect();
      return false;
    }
    Serial.println(" - Found our characteristic");

    // Read the value of the characteristic.
    if(pRemoteCharacteristic->canRead()) {
      std::string value = pRemoteCharacteristic->readValue();
      Serial.print("The characteristic value was: ");
      Serial.println(value.c_str());
    }

    if(pRemoteCharacteristic->canNotify())
      pRemoteCharacteristic->registerForNotify(notifyCallback);

    connected = true;

	return true;
}
/**
 * Scan for BLE servers and find the first one that advertises the service we are looking for.
 */
class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
 /**
   * Called for each advertising BLE server.
   */
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    Serial.print("BLE Advertised Device found: ");
    Serial.println(advertisedDevice.toString().c_str());

    // We have found a device, let us now see if it contains the service we are looking for.
    if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(serviceUUID)) {

      BLEDevice::getScan()->stop();
      myDevice = new BLEAdvertisedDevice(advertisedDevice);
      doConnect = true;
      doScan = true;

    } // Found our server
  } // onResult
}; // MyAdvertisedDeviceCallbacks

void setup() {
  Serial.begin(115200);
  Serial.println("Starting Arduino BLE Client application...");
  BLEDevice::setCustomGapHandler(my_gap_event_handler);
  BLEDevice::setCustomGattsHandler(my_gatts_event_handler);
  BLEDevice::setCustomGattcHandler(my_gattc_event_handler);

  BLEDevice::init("");

  // Retrieve a Scanner and set the callback we want to use to be informed when we
  // have detected a new device.  Specify that we want active scanning and start the
  // scan to run for 5 seconds.
  BLEScan* pBLEScan = BLEDevice::getScan();
  pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());
  pBLEScan->setInterval(2000);
  pBLEScan->setWindow(1500);
  pBLEScan->setActiveScan(true);
  pBLEScan->start(20, false);
} // End of setup.

// This is the Arduino main loop function.
void loop() {

  // If the flag "doConnect" is true then we have scanned for and found the desired
  // BLE Server with which we wish to connect.  Now we connect to it.  Once we are
  // connected we set the connected flag to be true.
  if (doConnect == true) {
    if (connectToServer()) {
      Serial.println("We are now connected to the BLE Server.");
    } else {
      Serial.println("We have failed to connect to the server; there is nothin more we will do.");
    }
    doConnect = false;
  }

  // If we are connected to a peer BLE Server, update the characteristic each time we are reached
  // with the current time since boot.
  if (connected) {
    String newValue = "Time since boot: " + String(millis()/1000);
    Serial.println("Setting new characteristic value to \"" + newValue + "\"");

    // Set the characteristic's value to be the array of bytes that is actually a string.
    pRemoteCharacteristic->writeValue(newValue.c_str(), newValue.length());
  }else if(doScan){
    BLEDevice::getScan()->start(0);  // this is just eample to start scan after disconnect, most likely there is better way to do it in arduino
  }

  delay(1000); // Delay a second between loops.
} // End of loop

Log:

rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9220
load:0x40080400,len:6300
entry 0x400806a4
Starting Arduino BLE Client application...
[D][BLEScan.cpp:204] start(): >> start(duration=20)
[D][FreeRTOS.cpp:165] take(): Semaphore taking: name: ScanEnd (0x3ffdf3ec), owner: <N/A> for start
[D][FreeRTOS.cpp:174] take(): Semaphore taken:  name: ScanEnd (0x3ffdf3ec), owner: start
[D][BLEScan.cpp:236] start(): << start()
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 2
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 7
[D][BLEAdvertisedDevice.cpp:424] setRSSI(): - setRSSI(): rssi: -48
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x01 (), length: 1, data: 1a
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0xff (), length: 9, data: 4c001005031c8bb496
[D][BLEAdvertisedDevice.cpp:401] setManufacturerData(): - manufacturer data: 4c001005031c8bb496
BLE Advertised Device found: Name: , Address: 5f:93:95:3c:d5:a6, manufacturer data: 4c001005031c8bb496
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEAdvertisedDevice.cpp:424] setRSSI(): - setRSSI(): rssi: -58
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x01 (), length: 1, data: 1a
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0xff (), length: 9, data: 4c001005031c0ba1c9
[D][BLEAdvertisedDevice.cpp:401] setManufacturerData(): - manufacturer data: 4c001005031c0ba1c9
BLE Advertised Device found: Name: , Address: 79:3a:d0:27:3f:1e, manufacturer data: 4c001005031c0ba1c9
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 79:3a:d0:27:3f:1e, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEScan.cpp:106] handleGAPEvent(): Ignoring 5f:93:95:3c:d5:a6, already seen it.
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[D][BLEAdvertisedDevice.cpp:424] setRSSI(): - setRSSI(): rssi: -66
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x01 (), length: 1, data: 06
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x02 (), length: 4, data: 1418f0ff
[D][BLEAdvertisedDevice.cpp:453] setServiceUUID(): - addServiceUUID(): serviceUUID: 00001814-0000-1000-8000-00805f9b34fb
[D][BLEAdvertisedDevice.cpp:453] setServiceUUID(): - addServiceUUID(): serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x09 (), length: 15, data: 4d696c6573746f6e65506f64204636
[D][BLEAdvertisedDevice.cpp:413] setName(): - setName(): name: MilestonePod F6
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x12 (), length: 4, data: 10002000
[D][BLEAdvertisedDevice.cpp:349] parseAdvertisement(): Unhandled type: adType: 18 - 0x12
[D][BLEAdvertisedDevice.cpp:253] parseAdvertisement(): Type: 0x0a (), length: 1, data: 00
[D][BLEAdvertisedDevice.cpp:484] setTXPower(): - txPower: 0
BLE Advertised Device found: Name: MilestonePod F6, Address: 50:33:8b:1c:ef:f6, serviceUUID: 00001814-0000-1000-8000-00805f9b34fb, txPower: 0
[D][BLEScan.cpp:259] stop(): >> stop()
[D][BLEScan.cpp:271] stop(): << stop()
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 3
[I][BLE_client.ino:59] my_gap_event_handler(): custom gap event handler, event: 18
Forming a connection to 50:33:8b:1c:ef:f6
[D][BLEDevice.cpp:62] createClient(): >> createClient
[D][BLEDevice.cpp:68] createClient(): << createClient
 - Created client
[D][BLEClient.cpp:103] connect(): >> connect(50:33:8b:1c:ef:f6)
[I][BLEDevice.cpp:596] addPeerDevice(): add conn_id: 0, GATT role: client
[D][FreeRTOS.cpp:165] take(): Semaphore taking: name: RegEvt (0x3ffdf8f8), owner: <N/A> for connect
[D][FreeRTOS.cpp:174] take(): Semaphore taken:  name: RegEvt (0x3ffdf8f8), owner: connect
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 0
[D][FreeRTOS.cpp:165] take(): Semaphore taking: name: OpenEvt (0x3ffdfa84), owner: <N/A> for connect
[D][FreeRTOS.cpp:174] take(): Semaphore taken:  name: OpenEvt (0x3ffdfa84), owner: connect
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEDevice.cpp:580] updatePeerDevice(): update conn_id: 4, GATT role: client
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 40
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:136] connect(): << connect(), rc=1
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 2
 - Connected to server
[D][BLEClient.cpp:383] getService(): >> getService: uuid: 00001814-0000-1000-8000-00805f9b34fb
[D][BLEClient.cpp:419] getServices(): >> getServices
[D][BLEClient.cpp:78] clearServices(): >> clearServices
[D][BLEClient.cpp:85] clearServices(): << clearServices
[D][FreeRTOS.cpp:165] take(): Semaphore taking: name: SearchCmplEvt (0x3ffdfd14), owner: <N/A> for getServices
[D][FreeRTOS.cpp:174] take(): Semaphore taken:  name: SearchCmplEvt (0x3ffdfd14), owner: getServices
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:32] BLERemoteService(): >> BLERemoteService()
[D][BLERemoteService.cpp:40] BLERemoteService(): << BLERemoteService()
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 7
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:32] BLERemoteService(): >> BLERemoteService()
[D][BLERemoteService.cpp:40] BLERemoteService(): << BLERemoteService()
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 7
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:32] BLERemoteService(): >> BLERemoteService()
[D][BLERemoteService.cpp:40] BLERemoteService(): << BLERemoteService()
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 7
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:32] BLERemoteService(): >> BLERemoteService()
[D][BLERemoteService.cpp:40] BLERemoteService(): << BLERemoteService()
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 7
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:32] BLERemoteService(): >> BLERemoteService()
[D][BLERemoteService.cpp:40] BLERemoteService(): << BLERemoteService()
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 7
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:32] BLERemoteService(): >> BLERemoteService()
[D][BLERemoteService.cpp:40] BLERemoteService(): << BLERemoteService()
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 7
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLEClient.cpp:436] getServices(): << getServices
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 6
[D][BLEClient.cpp:396] getService(): << getService: found the service with uuid: 00001814-0000-1000-8000-00805f9b34fb
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
 - Found our service
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteService.cpp:168] retrieveCharacteristics(): >> getCharacteristics() for service: 00001814-0000-1000-8000-00805f9b34fb
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 18
[D][BLERemoteService.cpp:199] retrieveCharacteristics(): Found a characteristic: Handle: 18, UUID: 00002a53-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:43] BLERemoteCharacteristic(): >> BLERemoteCharacteristic: handle: 18 0x18, uuid: 00002a53-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:256] retrieveDescriptors(): >> retrieveDescriptors() for characteristic: 00002a53-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:286] retrieveDescriptors(): Found a descriptor: Handle: 19, UUID: 00002902-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:300] retrieveDescriptors(): << retrieveDescriptors(): Found 1 descriptors.
[D][BLERemoteCharacteristic.cpp:51] BLERemoteCharacteristic(): << BLERemoteCharacteristic
[D][BLERemoteService.cpp:199] retrieveCharacteristics(): Found a characteristic: Handle: 21, UUID: 00002a54-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:43] BLERemoteCharacteristic(): >> BLERemoteCharacteristic: handle: 21 0x21, uuid: 00002a54-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:256] retrieveDescriptors(): >> retrieveDescriptors() for characteristic: 00002a54-0000-1000-8000-00805f9b34fb
[E][BLERemoteCharacteristic.cpp:280] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[D][BLERemoteCharacteristic.cpp:300] retrieveDescriptors(): << retrieveDescriptors(): Found 0 descriptors.
[D][BLERemoteCharacteristic.cpp:51] BLERemoteCharacteristic(): << BLERemoteCharacteristic
[D][BLERemoteService.cpp:199] retrieveCharacteristics(): Found a characteristic: Handle: 23, UUID: 00002a5d-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:43] BLERemoteCharacteristic(): >> BLERemoteCharacteristic: handle: 23 0x23, uuid: 00002a5d-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:256] retrieveDescriptors(): >> retrieveDescriptors() for characteristic: 00002a5d-0000-1000-8000-00805f9b34fb
[E][BLERemoteCharacteristic.cpp:280] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[D][BLERemoteCharacteristic.cpp:300] retrieveDescriptors(): << retrieveDescriptors(): Found 0 descriptors.
[D][BLERemoteCharacteristic.cpp:51] BLERemoteCharacteristic(): << BLERemoteCharacteristic
[D][BLERemoteService.cpp:199] retrieveCharacteristics(): Found a characteristic: Handle: 25, UUID: 00002a55-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:43] BLERemoteCharacteristic(): >> BLERemoteCharacteristic: handle: 25 0x25, uuid: 00002a55-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:256] retrieveDescriptors(): >> retrieveDescriptors() for characteristic: 00002a55-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:286] retrieveDescriptors(): Found a descriptor: Handle: 26, UUID: 00002902-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:300] retrieveDescriptors(): << retrieveDescriptors(): Found 1 descriptors.
[D][BLERemoteCharacteristic.cpp:51] BLERemoteCharacteristic(): << BLERemoteCharacteristic
[D][BLERemoteService.cpp:199] retrieveCharacteristics(): Found a characteristic: Handle: 28, UUID: 0000ff00-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:43] BLERemoteCharacteristic(): >> BLERemoteCharacteristic: handle: 28 0x28, uuid: 0000ff00-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:256] retrieveDescriptors(): >> retrieveDescriptors() for characteristic: 0000ff00-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:286] retrieveDescriptors(): Found a descriptor: Handle: 29, UUID: 00002902-0000-1000-8000-00805f9b34fb
[D][BLERemoteCharacteristic.cpp:300] retrieveDescriptors(): << retrieveDescriptors(): Found 1 descriptors.
[D][BLERemoteCharacteristic.cpp:51] BLERemoteCharacteristic(): << BLERemoteCharacteristic
[D][BLERemoteService.cpp:215] retrieveCharacteristics(): << getCharacteristics()
 - Found our characteristic
[D][BLERemoteCharacteristic.cpp:444] registerForNotify(): >> registerForNotify(): Characteristic: uuid: 0000ff00-0000-1000-8000-00805f9b34fb, handle: 28 0x1c, props: broadcast: 0, read: 0, write_nr: 0, write: 0, notify: 1, indicate: 0, auth: 0
[D][FreeRTOS.cpp:165] take(): Semaphore taking: name: RegForNotifyEvt (0x3ffe3bdc), owner: <N/A> for registerForNotify
[D][FreeRTOS.cpp:174] take(): Semaphore taken:  name: RegForNotifyEvt (0x3ffe3bdc), owner: registerForNotify
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteCharacteristic.cpp:329] getDescriptor(): >> getDescriptor: uuid: 00002902-0000-1000-8000-00805f9b34fb
[D][BLEClient.cpp:165] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteCharacteristic.cpp:333] getDescriptor(): << getDescriptor: found
[W][BLE_client.ino:47] my_gattc_event_handler(): custom gattc event handler, event: 38
[D][BLERemoteDescriptor.cpp:138] writeValue(): >> writeValue: handle: 29, uuid: 00002902-0000-1000-8000-00805f9b34fb
[D][BLERemoteDescriptor.cpp:157] writeValue(): << writeValue
[D][BLEDevice.cpp:150] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[D][BLERemoteCharacteristic.cpp:484] registerForNotify(): << registerForNotify()
Guru Meditation Error: Core  1 panic'ed (IllegalInstruction). Exception was unhandled.

Decoded stack trace
PC: 0x400d1eff: loop() at C:\Users\AKAEM\Documents\Arduino\hardware\espressif\esp32\cores\esp32/WString.h line 291
EXCVADDR: 0x00000000

Decoding stack results
0x400d1eff: loop() at C:\Users\AKAEM\Documents\Arduino\hardware\espressif\esp32\cores\esp32/WString.h line 291
0x40190660: loopTask(void*) at C:\Users\AKAEM\Documents\Arduino\hardware\espressif\esp32\cores\esp32\main.cpp line 17
0x40092ded: vPortTaskWrapper at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/freertos/port.c line 143

If I decode the exception with the (newly) build in function of VisualMicro I got this exception decoded:

   Error
   : ?? ??:0
   : ?? ??:0
   : ?? ??:0
   0x400d1eff StringSumHelper
   :\Users\AKAEM\Documents\Arduino\My Examples\BLE_client/BLE_client.ino:175
   : ?? ??:0
   : ?? ??:0
   :17 (discriminator 1):::0x40190660:loopTask(void*)
   :355 (discriminator 1):::0x40092ded:vPortTaskWrapper

0x400d1eff StringSumHelper = PC: 0x400d1eff: loop() at C:\Users\AKAEM\Documents\Arduino\hardware\espressif\esp32\cores\esp32/WString.h line 291

Did anyone solve this issue? I'm getting exactly the same crash at registerForNotify().

rst:0x8 (TG1WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1344
load:0x40078000,len:13836
load:0x40080400,len:3608
entry 0x400805f0
I'm Receving this type error can someone help me out


Simulator Scenario SchedulingSimulation supports in Web UI

Problem
In #140, #159, and #184, our simulator would be big changes.
But the current frontend is not suited for the new features.

Possible Issues
I think that we need to decide on two broad problems, at least.

Result view

We must implement a new view to see some scenario results.
The data provide from new features are completely different from the current.

Method to choose the simulator, URL, or port

The data provided by new features allow us to run multiple simulators.
Current frontend can only display data on one URL (simulator)

So, looking for good ideas!
We should have a little discussion before creating some frontend's KEP.

/kind feature

+1: Nice idea!
At least, it would be useful for users to have a UI that resembles a web browser.

/area web

/remove-lifecycle rotten
/lifecycle frozen


Provide Jakarta compatible build

JakartaEE 9.x is available which uses a new package jakarta instead of javax. Therefore the currently available jars are not compatible with runtimes based on JakartaEE 9.x.
A possible solution would be the usage of the eclipse transformer plugin, which produces an additional jar file with updated imports, etc.

            <plugin>
                <groupId>org.eclipse.transformer</groupId>
                <artifactId>org.eclipse.transformer.maven</artifactId>
                <version>0.2.0</version>
                <executions>
                    <execution>
                        <id>jakarta-ee</id>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <phase>package</phase>
                        <configuration>
                            <classifier>jakarta</classifier>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

Has there been any progress on this? It has been more than a year since Jakarta EE 9 was released.

The change is very easy, but once you make it. the version is not valid for systems with older libraries. I don't know how to do it in a sustainable way.
As I needed this in an app running in Tomcat 10, I made a fork and did the changes. I worked fine after I managed to fix a problem about the organization of resources inside the resulting ZIP.

I need this jakarta.servlet compliant dependency too.
For now i will duplicate the 2 dummy Auth/ServletUtils classes on my side and manually change the servlet pkg name, but it would be better to avoid such fork.
I tried a workaround with zero java code impact in #379 to be 100% retro-compatible, please let me know if such trick could ever go upstream to avoid maintaining the dummy fork (any advice to improve are welcome).

I think it would be ideal if java-saml refactored out all direct dependencies on javax.servlet.* and jakarta.servlet.*.
Meaning, replace all usages of HttpServletRequest and HttpServletResponse with a couple of new interfaces, say com.onelogin.saml2.http.HttpRequest and com.onelogin.saml2.http.HttpResponse. These new interfaces would define the minimum set of methods required for java-saml to work - surely not every method in the existing HttpServletRequest and HttpServletResponse interfaces is required for java-saml?
With that swap in place, the consumers of this library could provide their own HttpRequest and HttpResponse implementations based on either javax.servlet.* or jakarta.servlet.* (whatever Java/Jakarta EE implementation they happen to be using).
In theory, this would also allow java-saml to be used outside of servlet containers, e.g., what several of us we're hoping for in https://github.com/onelogin/java-saml/pull/115.
And FWIW, I understand what I'm proposing is effectively a backwards incompatible change. But it definitely seems like this is the right seam on which java-saml should divorce itself from specific servlet implementations and allow the user of the library to choose by providing their own HttpRequest and HttpResponse implementation.
CC // @pitbulk

Any chance for going forward with than anytime soon? Payara Community plans to work only with jakarta starting from version 6 so it will be crucial to make it usable with payara community.

@pitbulk ping?
We also ran into this, and now have to use a custom branch. #379 looks like a reasonable solution for this...

Hi @veselov ,

#379 looks like a reasonable solution for this...
I made this PR to share a quickwin workaround without in-depth code modification, but i totally agree with the @markkolich 's comment above: all these specific servlet-like dependancies should move out of the highlevel helper methods of the project, and to be replaced by samples to load a generic "onelogin servlet" from all other servlets-like object (javax, jakarta, jersey, ...): Basically, the underlying code doesn't need a real servlet but only some fields (url, http verb, ...) and a callback for the case when there is an automatic redirection

HI all, we need to work on a long term solution.
Is https://github.com/SAML-Toolkits/java-saml/pull/379 ok or should we go for the solution proposed by @markkolich?

@pitbulk @veselov @mauromol @lounagen here's my proposed solution: https://github.com/SAML-Toolkits/java-saml/pull/395
In https://github.com/SAML-Toolkits/java-saml/pull/395 I ripped out all usages of the servlet API and completely divorced core and toolkit from anything Servlet specific. The consumer of the library has to provide their own HttpRequest and HttpResponse implementations. For convenience, I introduced java-saml-servlet-javax and java-saml-servlet-jakarta which are working (sample?) implementations for javax.* and jakarta.* containers.
Consumers outside of a servlet container can provide their own implementations (Spray, Akka, Play, etc.) of HttpRequest and HttpResponse.
Additional notes are in the PR.
Thoughts and feedback are very welcome.

Until a jakarta version is out, i have a workaround thats quite easy to use and works well.

add dependency on https://mvnrepository.com/artifact/org.apache.felix/org.apache.felix.http.wrappers/1.1.2
use HttpServletRequest / Response from that library as wrapper for the jakarta counterparts.

Those wrapper classes wire all methods from jakarta to javax namespace used in java-saml.


Any plans to open source?

Hi there - great project! Looks like the actual source is kept offline and only the build files are being added to this repo. If you open source it, the community can help make it better. Any plans to do that?

Thanks! The plan is to release the source later this year.
To be able to better facilitate pull requests and new ideas I want to have more unit tests and need to restructure the repository. Without that, it'll cost way too much time to support incoming improvements/suggestions so therefore I've currently only released the build targets.

I understand, I’m trying to get this done ASAP but it’s not the only thing on my plate. I’d rather not have pull request than have pull requests I can’t review, judging from the issue quality so far a simple entry in the readme would not be enough.

I'd like to throw in my two cents here. I'm currently implementing filepond into one of our projects. The UI is really nice.
However not being able to look at the source makes things really difficult from time to time and also leaves a bad aftertaste, because it seems like you somehow don't want this to be open source at all. I can understand your decision regarding pull requests, but I'd really like you to reconsider this, I think not having the source available really hinders the progress of this great project.

I hear you @Sopamo

because it seems like you somehow don't want this to be open source at all

This is not helping. If I didn't want it to be open source I would've stated that in an earlier comment.

After reading my comment again I realized it didn't sound as I meant it. I tried to explain, that without reading your comment on this issue it seems like you don't want this project to be open source. The problem is, that almost no one will ever read your comment here, so people will wonder why the project isn't open source and probably won't come to the correct conclusion. Maybe you could at least link this issue in the readme?

@Sopamo Excellent idea I will do that right away.
Thanks for building a Laravel FilePond server!

I'm happy to support this great project :) I actually wanted to wait a bit to test the code more before telling you about it and maybe asking if you could link it in the readme, so Laravel people would find it quickly.

@Sopamo If you wish I can remove the link (I added it to the README as well, hadn't read your reply).

It's fine I guess, the code isn't very complex and works fine for our project.
Sorry guys, I'll stop spamming this issue now with irrelevant comments :)

I've started with open sourcing the plugins, they should all be open source in the next few weeks.
https://github.com/pqina/filepond-plugin-file-encode


Notification when Proxy has started and stopped

I have a process that depends on the proxy being fully available asynchronously. . Can we add notification events such as Proxy.onStart and Proxy.onStop events?

Pull request. (https://github.com/chhsiao90/nitmproxy/pull/38)


silent failure with xsd:dateTime

In trying to kludge around #22, I created my own prov-based properties. The startedAtTime and endedAtTime properties of activity have type xsd:dateTime, but sbol_factory apparently doesn't know how to handle that. Instead, it silently does not create the properties.

This should be straightforward to fix.


Upgrade to version 3

Kindly support version-3 of my API
I know it's not my duty to update you guys but thought you might be busy.
Within next 2 weeks Version 2 will be gone, reduced to atoms as I said last month that Version 4 will be end for version 2
So please

#22 fixed this.


Add free bsd firewall

@sharsonia any plan to update it?

feel free to create it when it is ready for review

@sharsonia, can @somil55 continue on this PR?

@sharsonia, can @somil55 continue on this PR?

@squirrelsc - Will update this today and test it out

@sharsonia @somil55 any update on this PR? If it's useful, can we merge it with a priority?


allow rewriting py file

Is your feature request related to a problem? Please describe.
while using the vetiver_write_app function, it does not allow rewriting the file if the file already exists
Describe the solution you'd like
Allow rewriting the file
Describe alternatives you've considered
For now, we manually delete the file first before rerunning the function.
Additional context

Thank you for this feedback 🙌  I had chosen to not allow rewriting to not have people accidentally rewriting/deploying app files without noticing. However, I can add an argument to allow for rewriting, so users can chose to turn off this safety net!

I looped this into #65 since vetiver_write_app (now vetiver.write_app) got a refresh. Setting overwrite=True should give you the desired behavior!


Disable the Sentry transactions tracing

https://helsinkicity.slack.com/archives/C01DWA87NJC/p1651312291390169

@channel liikuntahelsinki on hakannut juuri vaihtuneen Sentryn laskutuskauden koko 100k eventin quotan täyteen päivässä - katsokaa mikä pissii

Codecov Report

Merging #141 (7c0ec68) into main (809d975) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #141   +/-   ##
=======================================
  Coverage   65.98%   65.98%
=======================================
  Files         133      133
  Lines        2099     2099
  Branches      481      506   +25
=======================================
  Hits         1385     1385
  Misses        707      707
  Partials        7        7

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 809d975...7c0ec68. Read the comment docs.


fix: update combobox design to latest spec

Also fixes a regression on the invalid state notification
PR Checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)
[ ] If applicable, have a visual design approval

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] clarity.design website / infrastructure changes
[x] Other... Please describe: Update to latest design spec

What is the current behavior?
Currently the pills are designed to design spec minus one - based on the labels styles
Issue Number: N/A
What is the new behavior?
New style adheres to latest specs from design team, with smaller rounded borders, overall component border, different color scheme.
Does this PR introduce a breaking change?

[ ] Yes
[ ] No

Other information

@Jinnie
I'm assuming this won't be going into tomorrow's v4 release. That's fine, if so. Just verifying.

@Jinnie
I'm assuming this won't be going into tomorrow's v4 release. That's fine, if so. Just verifying.

No, I'll get the feedback from design team tomorrow, then I will need to update the site with the new screenshots and port it to v4. I don't think we can do it for tomorrow/today.