refactor: replace xxh3 with murmur3

This PR

Replace xxh3 with murmur3 in fractional evaluation's bucketing algorithm.
Related Issues

Part 1 of #843.
Notes

Follow-up Tasks

How to test

Hey @craigpastro, thanks for working on this! Could you please update the technical section in the docs to reference to murmur3 instead of xxh3?
https://github.com/open-feature/flagd/blob/main/docs/configuration/fractional_evaluation.md#fractional-evaluation-technical-description

👋  @ beeme1mr. Thank you. I've updated the docs and linked to the reference implementation. I don't know if that is the right thing to link to, but I didn't find anything better.

👋 @Kavindu-Dodan. Thank you for the review and for noticing that the documentation will conflict. I can rebase if #833 is merged first.

I think this is causing the build failure.


Default to production builds

🚀 Feature request
I know this is tough due to it being a breaking change with the ecosystem, but I think ng build should do a production build by default.
Command (mark with an x)
- [x] build

Description
Most of the time, developers should be building for production, and serving for development. It would save a lot of time overall if ng build defaulted to production mode.
Describe the solution you'd like
During the update process to 9 (or some other version), prompt the user if they want to keep the current behavior or migrate to the new behavior, and then default all new projects with ng new to production by default.
Describe alternatives you've considered
None

We actually followed this exact reasoning in 1.6, and some users were caught by surprise in a nasty way (https://github.com/angular/angular-cli/issues/10191), prompting us to revert it (https://github.com/angular/devkit/commit/b9704dc16efcbdfa3ccad59056709331bb2c4f50).
I can't imagine doing this change now would go any better since users have been used to npm run build not doing prod builds for much longer now.

Based on our discussion today, the feature makes a lot of sense. We should be very explicit in the way we communicate that we're using --prod.

@mgechev Are you saying the right behavior is to default to non-prod or to production mode?

Build —> production by default
Serve —> development by default
When the changeover happens (9.0? Sooner?) the concept would be that in addition to messaging in the release notes, a informational message would be displayed on build execution to provide ample opportunity to inform the user of the change in behavior.

Using our current setup, we could make the default options for build in angular.json contain the production settings, and have a separate dev configuration for development settings. Then ng serve would target the dev configuration for its build options.
We'd also have to provide good messages in the CLI about this. Users are used to using the --prod flag.

Awesome! Perhaps we could consider an update schematic that adds a "legacyBuildDefault"= true somewhere to existing applications that update to help folks with the transition.

This requires documentation changes and communicating to the user.
v9 with Ivy might not be the right time for this.
@StephenFluin - It would be good to have a design doc for this.

Hi @mgechev
I think we should make ng build --prod as default in angular-cli v9, there are lots of new users don't even try this command and assert that angular's bundle size is too big.

Consensus seems to be that this is a good idea which we should do. At this point we pretty much just need to actually implement it.

We had some discussion today about how to actually make this happen as there is a lot more underlying complexity than is immediately obvious. Some general notes:
Currently angular.json has options for ng build and various configurations. The default options are generally interpreted as "development" while --configuration production contains the production options. This is what is set up with ng new, but technically a project can have any number of configurations which can mean whatever they want. Angular is very flexible in this regard, but it also means we can make less assumptions about what changing configurations will actually do to an arbitrary project.
To reference similar projects, vue build defaults to production, while vue serve defaults to development (--mode flag). Although I believe Angular is more configurable about what "production" and "development" actually means, so Vue has a stronger abstraction model here.
There is still more discussion to be had here, but our current thought process is:

Update existing projects' npm run build script to actually execute ng build --configuration production.

This should fix services like Heroku which run npm run build and will make them use production builds.
For comparison, create-react-app and Vue both make npm run build generate a production build.

Update ng new schematics such that ng build uses production options by default.

We'll also make a separate development configuration that just needs to be explicitly passed.
ng serve needs to be updated to use --configuration development by default if available.

Create an optional migration for existing projects to make the production configuration default and add a separate development configuration with the current options.

Making this migration optional avoids a potentially significant and painful migration of angular.json being forced on users.
Users who really want this behavior and are invested in the change are willing to run a migration to get it.

Deprecate ng * --prod in favor of ng * --configuration production.

This does have a few notable caveats:

npm run build will now be different than ng build.

This will certainly trip some people up who would expect these two commands to be mostly equivalent.

ng build semantics vary by project.

Since the migration is optional, at least some existing projects will continue to have ng build -> "development", while new projects will have ng build -> "production".
A new developer joining an Angular project has no good way of knowing what ng build will do without looking at the project setup (which requires non-trivial knowledge of Angular configurations) or just trying it and seeing what happens.

Existing education content will likely need to be updated.

ng new --defaults foo && cd foo && ng build will now be prod and --configuration production would be removed and likely fail validation.
(Note to content authors, pin your versions to protect yourself from changes like this! For example, don't write npm install @angular/cli, write npm install<EMAIL_ADDRESS>

The biggest question we don't have a good answer to is what should happen for ng build --watch? It is just an ng build command, so if we did nothing special, this would build for production, which is most likely not what the user wanted. Some possible thoughts:

Could make ng build --watch require a --configuration flag.

This could be a major breaking change in itself.
What to do for "use default configuration"?

Add an npm run watch script which executes: ng build --watch --configuration development

This is a simple fix which targets what most users wanted to do to begin with.
Requires users to stop typing ng build --watch, and instead do npm run watch, which many aren't likely to do.

ng build --watch uses --configuration development by default.

Could be confusing to users. For example, contrasting ng build --watch vs ng build --watch --configuration something is adding the something configuration but also removing the development configuration.

This is also a potential issue with ng serve using --configuration development by default.

Deprecate ng build --watch and add ng watch which defaults to --configuration development.

ng serve avoids this problem because it is a distinct command from ng build which has a different use case and can easily support different defaults. Watching files is also a development use case, distinct from building.
Promoting ng build --watch to ng watch allows us to treat that as a distinct use case with a different default just like ng serve.

ng build --watch can log whether it is building for development or production.

Because Angular configurations are so flexible, there's really no way to know if the user intended "development" or "production", so what exactly we log is a bit of a question.
Could just log the configurations being used, although "using default configuration" would generally mean "using production configuration", but most users would not interpret it that way.
Could look at some specific option usually associated with production builds (like optimization: true), but that may not always directly map to a production build.

Definitely still more discussion to be had, so we'll follow up with this next week (or maybe after the holidays).

Discussed this issue a little more today.
Regarding the watch issue, we decided the simplest solution is to just add npm run watch which executes ng build --watch --configuration development. This causes the least changes and introduces minimal confusion to users. NPM scripts are also generally well understood by web developers and the extra transparency means looking up "What does npm run build do for this project?" is a lot easier than "What does ng build --watch do for this project?"
The ng watch solution seemed to be the next most liked solution, however it is a bit heavyweight and we didn't want to fully commit to that just yet. Instead, we decided to move forward with just npm run watch for now with some documentation and release notes pointing users to prefer that. If we still see a lot of confusion in this area, we can reconsider a more comprehensive change around deprecating ng build --watch and migrating users to ng watch. (Personally I feel my suggestion of ng serve --dont did not receive the consideration it should have :laughing:.)
Also a quick correction to the previous comment: ng serve, would not use --configuration development by default, but rather --browserTarget app:build:development. The use of --browserTarget means that we don't default to --configuration development, which would add flag confusion around how to build the default configuration. --browserTarget is often a tripping point for a lot of new users, but that is a separate problem to making ng build build for production by default.
One other thought around --prod since this will likely be a significant breaking change for existing education content. When we initially restructure angular.json to make the default configuration build for production, we should add a no-op production configuration. This would allow --configuration production and --prod to still work, even though they won't actually do anything. At that point we can add a deprecation message to --prod explaining the change and advising users to:

If this is an existing project, apply the optional migration and then drop --prod since it won't do anything.
If this is an existing project and they don't want to migrate, switch their command line to --configuration production.
If this is a new project, don't pass --prod, since it already doesn't do anything.

Hopefully this non-breaking warning should catch a lot of content authors and give them an opportunity to update their material before we fully remove --prod. We can remove the no-op production configuration as a later migration.
So with all that I think the list of changes for new projects (ng new) is:

Update angular.json default configuration for build to use production options.
Add a separate development configuration to angular.json builds.
Update angular.json default serve configuration to use --browserTarget app:build:development.
Add npm run watch which executes ng build --watch --configuration development.

And the list of changes for existing projects is:

Add migration to update npm run build to actually execute ng build --configuration production.

This should fix services like Heroku which run npm run build and will make them use production builds.

Add migration to add npm run watch which executes ng build --watch.
Add an optional migration for existing projects to make the existing production configuration the default and add a separate development configuration with the existing default options.

Should also update npm run build to just use ng build since that will now be prod by default.
Should also update npm run watch to use ng build --watch --configuration development.

Once those are ready we then need to:

Deprecate ng * --prod in favor of ng * --configuration production.

Clearly document for education content authors what is changing and how they should update their content.

Release all of the above in a major version.
Remove --prod option.
Wait until subsequent major version to release.

Related: https://github.com/angular/angular-cli/issues/12351


Worlds and height

i just found out about this project and really love it, especially since it's similar to an idea i myself had a while ago. but i have two questions:

often times virtual worlds like second life, wow, etc have multiple worlds/dimensions. how will decentraland handle this? child blockchains? sister blockchains? not at all? i think to have everything laid out on a single flat world sets a lot of limitations on what can be made.

how is height (or even depth) handled in decentraland? can everyone go into the sky as high as they want? can you even go underground? having a z-axis in the world/chain would make a lot of things more complicated. you'd need to purchase the tiles below/above you if you want to make very high projects. and what about rendering, light, etc? but i think not having a z-axis might create even more limitations. allowing everyone to go as high/low on their tile as they want to would create problems. setting a max height would create even more problems and limit users. and without allowing them to purchase space above/below them there's nothing they can do about it.

reposed this on the forum since it seems more appropriate there

Link: https://forum.decentraland.org/topic/5/worlds-and-height


Issue in reproducing the training loss vs learning rates curve

Hi,
First of all, thanks for sharing your work.
We tried to reproduce the expected behavior of muP, using ResNet18 and the CIFAR10, as provided in the main script of your repository. The idea was to launch a training, for multiple learning rates and width_mult, and get the minimum loss each time, as you did in your paper, to ensure that the best learning rate doesn't change with a different width_mult.
We modified a bit the main.py script, to skip the saving/loading of the  base shape file, as follows:
'''Train CIFAR10 with PyTorch.'''
import argparse
import os
from time import gmtime, strftime
import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
from mup import MuAdam, MuSGD, get_shapes, set_base_shapes
from copy import deepcopy
from mup.infshape import InfShape
from mup.shape import clear_dims, zip_infshapes
from torch.utils.tensorboard import SummaryWriter
import resnet

# Training
def train(epoch, net, writer):
#    from utils import progress_bar
    print('\nEpoch: %d' % epoch)
    net.train()
    train_loss = 0
    correct = 0
    total = 0
    for batch_idx, (inputs, targets) in enumerate(trainloader):
        inputs, targets = inputs.to(device), targets.to(device)
        optimizer.zero_grad()
        outputs = net(inputs)
        loss = criterion(outputs, targets)
        loss.backward()
        optimizer.step()

        train_loss += loss.item()
        _, predicted = outputs.max(1)
        total += targets.size(0)
        correct += predicted.eq(targets).sum().item()
    writer.add_scalar("Train/Loss", train_loss/(batch_idx+1), epoch)
    writer.add_scalar("Train/Acc", 100.*correct/total, epoch)

    return train_loss/len(trainloader)

def test(epoch, net, writer):
#    from utils import progress_bar
    global best_acc
    net.eval()
    test_loss = 0
    correct = 0
    total = 0
    with torch.no_grad():
        for batch_idx, (inputs, targets) in enumerate(testloader):
            inputs, targets = inputs.to(device), targets.to(device)
            outputs = net(inputs)
            loss = criterion(outputs, targets)

            test_loss += loss.item()
            _, predicted = outputs.max(1)
            total += targets.size(0)
            correct += predicted.eq(targets).sum().item()
    writer.add_scalar("Test/Loss", test_loss, epoch)
    writer.add_scalar("Test/Acc", 100.*correct/total , epoch)

    # Save checkpoint.
    acc = 100.*correct/total
    if acc > best_acc:
        print('Saving..')
        state = {
            'net': net.state_dict(),
            'acc': acc,
            'epoch': epoch,
        }
        if not os.path.isdir('checkpoint'):
            os.mkdir('checkpoint')
        torch.save(state, './checkpoint/ckpt.pth')
        best_acc = acc

    return test_loss/len(testloader), best_acc

# Custom method to skip save and load shapes
def get_base_shapes(base_shapes, delta_shapes):
    model_or_shapes = clear_dims(zip_infshapes(base_shapes, delta_shapes))
    if isinstance(model_or_shapes, nn.Module):
        sh = get_infshapes(model_or_shapes)
    elif isinstance(model_or_shapes, dict):
        sh = deepcopy(model_or_shapes)
    else:
        raise ValueError()
    sh = {k: s.base_shape() for k, s in sh.items()}
    return {k: InfShape.from_base_shape(v) for k, v in sh.items()}

if __name__ == "__main__":

    parser = argparse.ArgumentParser(description=''
    '''
    PyTorch CIFAR10 Training, with μP.
    To save base shapes info, run e.g.
        python main.py --save_base_shapes resnet18.bsh --width_mult 1
    To train using MuAdam (or MuSGD), run
        python main.py --width_mult 2 --load_base_shapes resnet18.bsh --optimizer {muadam,musgd}
    To test coords, run
        python main.py --load_base_shapes resnet18.bsh --optimizer sgd --lr 0.1 --coord_check
        python main.py --load_base_shapes resnet18.bsh --optimizer adam --lr 0.001 --coord_check
    If you don't specify a base shape file, then you are using standard parametrization, e.g.
        python main.py --width_mult 2 --optimizer {muadam,musgd}
    Here muadam (resp. musgd) would have the same result as adam (resp. sgd).
    Note that models of different depths need separate `.bsh` files.
    ''', formatter_class=argparse.RawTextHelpFormatter)

    parser.add_argument('--lr', default=0.1, type=float, help='learning rate')
    parser.add_argument('--resume', '-r', action='store_true', help='resume from checkpoint')
    parser.add_argument('--arch', type=str, default='resnet18')
    parser.add_argument('--optimizer', default='musgd', choices=['sgd', 'adam', 'musgd', 'muadam'])
    parser.add_argument('--epochs', type=int, default=150)
    parser.add_argument('--width_mult', type=float, default=1)
    parser.add_argument('--batch_size', type=int, default=128)
    parser.add_argument('--test_batch_size', type=int, default=128)
    parser.add_argument('--weight_decay', type=float, default=5e-4)
    parser.add_argument('--num_workers', type=int, default=2)
    parser.add_argument('--test_num_workers', type=int, default=2)
    parser.add_argument('--momentum', type=float, default=0.9)
    parser.add_argument('--seed', type=int, default=1111, help='random seed')

    args = parser.parse_args()

    root_dir = "/out/"

    device = 'cuda' if torch.cuda.is_available() else 'cpu'
    best_acc = 0  # best test accuracy
    start_epoch = 0  # start from epoch 0 or last checkpoint epoch

    # Set the random seed manually for reproducibility.
    torch.manual_seed(args.seed)

    print('==> Preparing data..')
    transform_train = transforms.Compose([
        transforms.RandomCrop(32, padding=4),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
    ])

    transform_test = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
    ])

    trainset = torchvision.datasets.CIFAR10(
        root='../dataset', train=True, download=True, transform=transform_train)
    trainloader = torch.utils.data.DataLoader(
        trainset, batch_size=args.batch_size, shuffle=True, num_workers=args.num_workers)

    testset = torchvision.datasets.CIFAR10(
        root='../dataset', train=False, download=True, transform=transform_test)
    testloader = torch.utils.data.DataLoader(
        testset, batch_size=args.test_batch_size, shuffle=False, num_workers=args.test_num_workers)

    classes = ('plane', 'car', 'bird', 'cat', 'deer',
            'dog', 'frog', 'horse', 'ship', 'truck')

    # Model
    print('==> Building model..')
    net = getattr(resnet, args.arch)(wm=args.width_mult)
    net = net.to(device)
    if args.optimizer in ["musgd","muadam"]:
        print(f'using muP Parametrization')
        base_shapes = get_shapes(net)
        delta_shapes = get_shapes(getattr(resnet, args.arch)(wm=args.width_mult/2))
        dict_infshape = get_base_shapes(base_shapes, delta_shapes)

        set_base_shapes(net, dict_infshape)
    else:
        print(f'using Standard Parametrization')
        set_base_shapes(net, None)

    if args.resume:
        # Load checkpoint.
        print('==> Resuming from checkpoint..')
        assert os.path.isdir('checkpoint'), 'Error: no checkpoint directory found!'
        checkpoint = torch.load('./checkpoint/ckpt.pth')
        net.load_state_dict(checkpoint['net'])
        best_acc = checkpoint['acc']
        start_epoch = checkpoint['epoch']

    criterion = nn.CrossEntropyLoss()
    if args.optimizer == 'musgd':
        optimizer = MuSGD(net.parameters(), lr=args.lr,
                            momentum=args.momentum,
                            weight_decay=args.weight_decay)
    elif args.optimizer == 'muadam':
        optimizer = MuAdam(net.parameters(), lr=args.lr)
    elif args.optimizer == 'sgd':
        optimizer = optim.SGD(net.parameters(), lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)
    elif args.optimizer == 'adam':
        optimizer = optim.Adam(net.parameters(), lr=args.lr)
    else:
        raise ValueError()
    scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=args.epochs)

    tb_time = strftime("%Y-%m-%d-%H:%M:%S", gmtime())
    sub_dir = root_dir + tb_time + "-" + str(args.arch) +  "-" + str(args.lr) + "-" + str(args.width_mult)

    os.makedirs(sub_dir, exist_ok = True)

    writer = SummaryWriter(sub_dir)

    for epoch in range(start_epoch, start_epoch+args.epochs):
        best_train_loss = train(epoch, net, writer)
        best_test_loss, best_acc = test(epoch, net, writer)
        scheduler.step()

    writer.add_hparams({"Epochs": args.epochs, "Width": args.width_mult, "BatchSize": args.batch_size}, {"Test/Score": best_acc})

Then,  for each width multiplier wm from 1 to 5, we launched the following bash scripts, which train the models for a set of learning rates.
In muP mode:
#!/bin/bash
wm=5
cd /exp/ResNetTest
pip3 install mup
for lr in 6.10351562e-05 8.13100441e-05 1.08319921e-04 1.44302039e-04 \
          1.92236834e-04 2.56094789e-04 3.41165320e-04 4.54494899e-04 \
          6.05470725e-04 8.06598270e-04 1.07453712e-03 1.43148090e-03 \
          1.90699561e-03 2.54046859e-03 3.38437101e-03 4.50860411e-03 \
          6.00628919e-03 8.00148094e-03 1.06594430e-02 1.42003369e-02 \
          1.89174582e-02 2.52015307e-02 3.35730700e-02 4.47254987e-02 \
          5.95825832e-02 7.93749499e-02 1.05742019e-01 1.40867801e-01 \
          1.87661798e-01 2.50000000e-01
do
    echo "Training for wm = ${wm} and lr = ${lr}"
    python3 main.py --lr=$lr --epochs=150 --batch_size=128 --num_workers=4 --seed=1111 --width_mult=$wm
done

In SP mode:
#!/bin/bash
wm=5
cd /exp/ResNetTest
pip3 install mup
for lr in 2.56094789e-04 4.54494899e-04 \
          8.06598270e-04 1.43148090e-03 \
          2.54046859e-03 4.50860411e-03 \
          8.00148094e-03 1.42003369e-02 \
          2.52015307e-02 4.47254987e-02 \
          7.93749499e-02 1.40867801e-01 \
          2.50000000e-01
do
    echo "Training for wm = ${wm} and lr = ${lr}"
    python3 main.py --lr=$lr --epochs=150 --batch_size=128 --num_workers=4 --seed=1111 --width_mult=$wm --optimizer='sgd'
done

Then, we get the minimum loss and plot the two curves (loss vs lr) : one with mup, one without.
With muP :

Without muP :

As you can see on the two figures, there is no visible difference between the two scenarios: In both case, minima are aligned except for those with wm=1
Do you have an idea why it is happening ?
Thanks for your help

Thanks for reaching out, Nicolas!
There are two reasons:

SGD under SP is "less wrong" than Adam so it takes a much larger delta in width to see the benefit of muP;
Here, wm is linear in width, so your widest model is 5x wider than the narrowest model. This delta is quite small. In our Transformer experiments with Adam, width is varied from 128 to 8192.

Thanks for your reply Edward!
We understand that under SP our width multiplier range may be too small and that the LR alignment in SP may fail for larger width. However, what we don t understand is the misalignment under muP when comparing the curve for wm=1 with the other wm values. I thought that under muP, the LR alignment is guaranteed?

Adding to Edward, you can see from the plot in our paper that the difference on ResNet on CIFAR10 is not as drastic as on transformers. The width difference is 16x btw the largest and smallest model here.

In addition, if you tune the input weight and/or output weight learning rate separately from the global learning rate on the smallest model, then often you'll be able to see muP performing much better and the alignment better.

Thanks for your reply Edward!
We understand that under SP our width multiplier range may be too small and that the LR alignment in SP may fail for larger width. However, what we don t understand is the misalignment under muP when comparing the curve for wm=1 with the other wm values. I thought that under muP, the LR alignment is guaranteed?

The answer is twofold: 1) the alignment is only approximate, improving with width of the base model (the analogy is estimating the mean of a population by taking a large sample and calculating its average --- this average is only approximately the same as the population mean, and only when the sample size is large enough), and 2) this has to do with insufficient tuning of other hyperparameters like input/output LR like I mentioned. You can check out my reply here.
In particular,

The true hyperparameter space here is the very high dimensional space containing [learning rate, initialization] (we can insert multipliers here as well, but like I said, it is redundant) for every parameter tensor (weights, biases, gains, etc). If you were to tune all these hyperparameters and obtain the optimal combination, then this combination is guaranteed to be stable in some sense as you vary width (in muP). However, in practice, we may not want to tune that many hyperparameters because of resource constraints. So we combine hyperparameters (by e.g., tying learning rate for many weights together) until we have only a small number to tune. This essentially means that we are now focusing on a low dimensional slice of the true hyperparameter space --- that we guess should contain all the really good hyperparameters. The choices of hyperparameters we tuned in our paper exemplify the “low dimensional slice” we chose. These choices are based on our empirical experience tuning hyperparameters, but over time people may find better choices.

Thank you Greg for your detailed explanation!


refactor: ♻ make extract script work for any game

Added game name to URL. Should work for any game.
Tested for Skyrim SE, Cyberpunk 2077, Fallout 4.

Thanks, I didn't know you could do that, I will refactor the whole repo this week, hopefully, but thanks.


Fixes #2651 - replies in notifications not sending

Update the worker logic to stop the reply worker from hanging when sending a reply to a notification.

Merging in to get you some credit, but I'll be changing this in the Kotlin impl.


Properly support ottavas

This commit adds support for properly transposing pitches affected by an <octave> element that has a @plist attribute.  Support for @startid/@endid/@tstamp/@tstamp2 can be added later with the help of additional <key>s, but with those attributes it's harder to find out what pitches are affected by the <octave>.

I'm still not convinced. The @plist makes sense if you have multiple layers in one staff and only one is affected by the ottava sign. But for the visual appearence of the ottava line @startid  and @endid should be used.

I changed the template so now it doesn't have to use \ottava. This should simplify all and is more "MEI-like". Plz try, if it works for you.
As I produced a conflict though, I'll close this PR for now.
(We should rethink this whole <octave> business at Tours.)


bind_port is not being respected

Version 1.2.1 crawlera-headless-proxy-linux-amd64
OS: Debian 4.9.168-1+deb9u4
When setting the bind_port in any way (command line flags, environment vars or through a config file) it is not respected.
It is always set to 3129.
EDIT: This seems to occur when running more than one instance of crawlera-headless-proxy.

Sorry for the delay. I'll check

Hi @9seconds
Could it be because of the proxy api?
https://github.com/scrapinghub/crawlera-headless-proxy#proxy-api
Is there a way to set the port that it listens on?
Thanks!

@joshuasmall I've checked, everything works as intended. Basically, this application has 2 different ports: one for proxy api (which you access with your http client or browser) and another one for stats api (where you can see some json with stats)
$ ./crawlera-headless-proxy -a "$CRAWLERA_MYAUTH" -d -p 8888 -w 9999
DEBU[0000] TLS checksums.                                ca-cert=dcb3f998f0991455ef93583d4d6d49d71fa91e80 priv-key=a556e5cdb31e233ed4331610196c62010b9fa7cb
DEBU[0000] Listen on <IP_ADDRESS>:8888                      adblock-lists="[]" apikey=*** bindip=<IP_ADDRESS> bindport=8888 concurrent-connections=0 crawlera-host=proxy.crawlera.com crawlera-port=8010 debug=true direct-access-hostpath-regexps="[]" dont-verify-crawlera-cert=false no-auto-sessions=false proxy-api-ip=<IP_ADDRESS> proxy-api-port=9999 xheaders="map[]"
...

$ curl -x localhost:8888 -ik https://httpbin.org/headers
HTTP/1.1 200 OK
Date: Wed, 01 Apr 2020 06:12:29 GMT
Content-Length: 0

HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 01 Apr 2020 06:12:33 GMT
Content-Type: application/json
Content-Length: 532
X-Crawlera-Session:<PHONE_NUMBER>
X-Crawlera-Version: 1.43.0-a83f3a
access-control-allow-origin: *
access-control-allow-credentials: true
x-powered-by: Flask
x-processed-time: 0
x-upstream: httpbin-master_web
Proxy-Connection: close
Connection: close

{
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.5",
    "Forwarded": "for=*******;proto=https",
    "Host": "httpbin.org",
    "Proxy-Authorization": "Basic *****",
    "Referer": "https://httpbin.org/headers",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:74.0) Gecko/20100101 Firefox/74.0"
  }
}

$ curl localhost:9999/stats
{
  "requests_number": 4,
  "crawlera_requests": 5,
  "sessions_created": 1,
  "clients_connected": 0,
  "adblocked_requests": 0,
  "crawlera_errors": 2,
  "all_errors": 4,
  "overall_times": {
    "average": 3.8498335285,
    "minimal": 2.428097765,
    "maximal": 4.965745319,
    "median": 4.002745515,
    "standard_deviation": 0.9689998911970745,
    "percentiles": {}
  },
  "crawlera_times": {
    "average": 3.0794386718,
    "minimal": 0.99316086,
    "maximal": 4.96556832,
    "median": 3.532206065,
    "standard_deviation": 1.5978717257630468,
    "percentiles": {}
  },
  "uptime": 299
}

So, -p goes to proxy api port, -w - to stats api port
I close this ticket. Please feel free to reopen if you have any questions
And again, I'm really sorry for the delay in answering your questions 🙇

Hi @9seconds
Thank you for checking this for me.


AssertionError thrown from AbstractSailConnection when using Federation

When adding multiple repositories to a federation and evaluating a query which contains a NaryJoin the query is executed over multiple threads. This can result in AbstractSailConnection.startUpdate throwing an assertion error because multiple threads attempt to flush the same federation connection instance concurrently. This also results in the query failing to return the expected results.
Here is an example test which reproduces the problem. On my machine this test reproduces the issue maybe once in ten runs on average.
public void testAssertionErrorReproduction() {

        final ValueFactory vf = SimpleValueFactory.getInstance();
        final BNode address = vf.createBNode();
        final BNode anotherAddress = vf.createBNode();

        final ModelBuilder builder = new ModelBuilder();
        builder
            .setNamespace("ex", "http://example.org/")
            .subject("ex:Picasso")
                .add(RDF.TYPE, "ex:Artist")
                .add(FOAF.FIRST_NAME, "Pablo")
                .add("ex:homeAddress", address)
            .subject("ex:AnotherArtist")
                .add(RDF.TYPE, "ex:Artist")
                .add(FOAF.FIRST_NAME, "AnotherArtist")
                .add("ex:homeAddress", anotherAddress)
            .subject(address)
                .add("ex:street", "31 Art Gallery")
                .add("ex:city", "Madrid")
                .add("ex:country", "Spain")
            .subject(anotherAddress)
                .add("ex:street", "32 Art Gallery")
                .add("ex:city", "London")
                .add("ex:country", "UK");

        final Model model = builder.build();
        final Repository repo1 = new SailRepository(new MemoryStore());
        repo1.initialize();
        repo1.getConnection().add(model);

        final Repository repo2 = new SailRepository(new MemoryStore());
        repo2.initialize();

        final Federation fed = new Federation();
        fed.addMember(repo1);
        fed.addMember(repo2);

        final String ex = "http://example.org/";
        final String queryString =
                "PREFIX rdf: <" + RDF.NAMESPACE + ">\n" +
                "PREFIX foaf: <" + FOAF.NAMESPACE + ">\n" +
                "PREFIX ex: <" + ex + ">\n" +
                "select (count(?persons) as ?count) {\n" +
                "   ?persons rdf:type ex:Artist ;\n"
                + "          ex:homeAddress ?country .\n"
                + " ?country ex:country \"Spain\" . }";

        final SailRepository fedRepo = new SailRepository(fed);
        fedRepo.initialize();

        final SailRepositoryConnection fedRepoConn = fedRepo.getConnection();

        fedRepoConn.begin();
        final TupleQuery query = fedRepoConn.prepareTupleQuery(QueryLanguage.SPARQL, queryString);
        final TupleQueryResult eval = query.evaluate();
        if (eval.hasNext()) {
            final Value next = eval.next().getValue("count");
            assertEquals(1, ((Literal) next).intValue());
        }
        else {
            fail("No result");
        }

        fedRepoConn.commit();
    }

Here is the stack trace.
Exception in thread "rdf4j-federation-1" java.lang.AssertionError
at org.eclipse.rdf4j.sail.helpers.AbstractSailConnection.startUpdate(AbstractSailConnection.java:512)
at org.eclipse.rdf4j.sail.helpers.AbstractSailConnection.flush(AbstractSailConnection.java:388)
at org.eclipse.rdf4j.sail.helpers.AbstractSailConnection.flushPendingUpdates(AbstractSailConnection.java:929)
at org.eclipse.rdf4j.sail.helpers.AbstractSailConnection.getStatements(AbstractSailConnection.java:318)
at org.eclipse.rdf4j.sail.federation.AbstractFederationConnection$FederationTripleSource.getStatements(AbstractFederationConnection.java:400)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:538)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:234)
at org.eclipse.rdf4j.sail.federation.evaluation.FederationStrategy.evaluate(FederationStrategy.java:65)
at org.eclipse.rdf4j.sail.federation.evaluation.ParallelJoinCursor.run(ParallelJoinCursor.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I have managed to reproduce your issue by putting your code inside a loop. I then modified your code to correctly close all closeable objects and the issue is solved. See: https://github.com/eclipse/rdf4j/pull/1619

        @Test
	public void testAssertionErrorReproduction() {

		for (int i = 0; i < 100; i++) {

			ValueFactory vf = SimpleValueFactory.getInstance();
			BNode address = vf.createBNode();
			BNode anotherAddress = vf.createBNode();

			ModelBuilder builder = new ModelBuilder();
			builder
					.setNamespace("ex", "http://example.org/")
					.subject("ex:Picasso")
					.add(RDF.TYPE, "ex:Artist")
					.add(FOAF.FIRST_NAME, "Pablo")
					.add("ex:homeAddress", address)
					.subject("ex:AnotherArtist")
					.add(RDF.TYPE, "ex:Artist")
					.add(FOAF.FIRST_NAME, "AnotherArtist")
					.add("ex:homeAddress", anotherAddress)
					.subject(address)
					.add("ex:street", "31 Art Gallery")
					.add("ex:city", "Madrid")
					.add("ex:country", "Spain")
					.subject(anotherAddress)
					.add("ex:street", "32 Art Gallery")
					.add("ex:city", "London")
					.add("ex:country", "UK");

			Model model = builder.build();
			Repository repo1 = new SailRepository(new MemoryStore());
			repo1.init();
			try (RepositoryConnection connection = repo1.getConnection()) {
				connection.add(model);
			}

			Repository repo2 = new SailRepository(new MemoryStore());
			repo2.init();

			Federation fed = new Federation();
			fed.addMember(repo1);
			fed.addMember(repo2);

			String ex = "http://example.org/";
			String queryString = "PREFIX rdf: <" + RDF.NAMESPACE + ">\n" +
					"PREFIX foaf: <" + FOAF.NAMESPACE + ">\n" +
					"PREFIX ex: <" + ex + ">\n" +
					"select (count(?persons) as ?count) {\n" +
					"   ?persons rdf:type ex:Artist ;\n"
					+ "          ex:homeAddress ?country .\n"
					+ " ?country ex:country \"Spain\" . }";

			SailRepository fedRepo = new SailRepository(fed);
			fedRepo.init();

			try (SailRepositoryConnection fedRepoConn = fedRepo.getConnection()) {

				fedRepoConn.begin();
				TupleQuery query = fedRepoConn.prepareTupleQuery(QueryLanguage.SPARQL, queryString);
				try (TupleQueryResult eval = query.evaluate()) {
					if (eval.hasNext()) {
						Value next = eval.next().getValue("count");
						assertEquals(1, ((Literal) next).intValue());
					} else {
						fail("No result");
					}
				}

				fedRepoConn.commit();
			}
		}
	}

fyi repos should be shutdown() too

Thanks for taking a look. However, I tried a few runs of your code and it failed with the same stack trace again. Could you take another look please?

I upped the loop to 1000000 and it is still passing. Which version of RDF4J are you using?

Reproduced.

Affected line.

@ieurich thank you for your persistence :)

@ieurich this has now been fixed and will be released as part of 3.0.2.
I work on RDF4J on my spare time for free, so if you or your organisation finds RDF4J useful then I would appreciate if you considered sponsoring me on Github:  https://github.com/sponsors/hmottestad

@ieurich 3.0.2 is released with a fix for this issue: https://rdf4j.eclipse.org/news/2019/10/26/rdf4j-3.0.2-released/


Exclude Google Analytics on localhost

Hello @adkdev,
thank you for submitting this pull request. Shortly after opening this pull request Hugo's internal Disqus template has been updated and doesn't load on localhost anymore (see https://github.com/gohugoio/hugo/commit/2e1e4934b60ce8081a7f3a79191ed204f3098481).
Since the behavior should be the same I would prefer to use the Hugo's internal template. Nonethless, thanks for your initiative.
Cheers,
Digitalcraftsman


Kafka Connect REST API Security

I was trying to follow the documentation here to set up SSL encryption for Kafka Connect REST API. Has anyone successfully configured https with SSL certificates as explained in the documentation? I tried many combinations of possible values for the following parameters, but none of them worked.
listeners=https://myhost:8443
rest.advertised.listener=https
rest.advertised.host.name=<localhost>
rest.advertised.host.port=8083
listeners.https.ssl.client.auth=requested
listeners.https.ssl.truststore.location=/var/ssl/private/kafka.server.truststore.jks
listeners.https.ssl.truststore.password=test1234
listeners.https.ssl.keystore.location=/var/ssl/private/kafka.server.keystore.jks
listeners.https.ssl.keystore.password=test1234
listeners.https.ssl.key.password=test1234

It turns out that this set of configuration works on docker images confluentinc/cp-base and openjdk:8. It currently does work on ibmjava:jre-8 and ibmjava:sdk-8.


Added N-Queen problem code in cpp folder

Please review my PR and help me get one step closer to Hacktoberfest 2020.

Hey @atyant-yadav please merge my PR.


Problem with ColReorder, Date Range, Server Side

Hi, I have recently found a bug, when using your plugin in a server side table with ColReorder and a date range filter.
For example:  table with [1, 2, 3, 4], the 4th is the date range.

When initialized and then filter the date everything works fine.
Then I change to [1, 2, 4, 3] and try to filter again, this time the table filters the column 3 with date range filter.
Last I change back to [1, 2, 3, 4], everything seems to be messed up and the table tries to filter column 3 and 4 with the date range.

Meanwhile, I have forked your script and fixed for our needs, but you could check if there is more of that type. (https://github.com/dotpmrcunha/yadcf)

I looked at the line of code that you modified and did the same in yadcf way let me know if there are more places that need a fix , grab 0.8.9.beta.22


Create Foundation to Manage Nftoken Program

I don't want Glow to be the only organization with control over the Nftoken program and standard. I think it's a problem that Metaplex is a private company that has raised VC money and it has total power over the current NFT standard.
On Solana, you get a ton of power from owning a program since you can upgrade the program and change all the accounts that the program owns. If Metaplex wanted to, they could remove all of your NFTs from the blockchain.
I'd like to work with existing players in the ecosystem and include room for other parties:

Magic Eden
OpenSea
Formfunction
Solana Foundation

I see this working in a similar way to W3C and to how DNS is run.

as a part of removing monopoly from a "standard" being totally headed by one organization. I would love to take part and make artz.ai compatible with these standards.
we gotta start somewhere.


Fix title.

Replacing HTTP_REQUEST with HTTP_RESPONSE in the title.

@M3ikShizuka : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Failed prop type: Invalid child component [object Object]. Must be one of: Cover, Body, Footer

I'm using Theme UI with the pragma set in .babelrc like this:
{
  "presets": [["next/babel", {
    "preset-react": {
      "pragma": "jsx"
    }
  }]],
  "plugins": [
    [
      "@emotion/babel-plugin-jsx-pragmatic",
      {
        "module": "theme-ui",
        "import": "jsx",
        "export": "jsx"
      }
    ]
  ]
}

and I'm creating the following macro component:
import React from "react";
import Macro from "macro-components";
import Flex from "../Flex";
import Box from "../Box";
import Stack from "../Stack";

const Cover = ({ children }) => (
  <Box
    sx={{
      borderBottom: "thin",
      borderColor: "muted",
    }}
  >
    {children}
  </Box>
);

const Body = ({ children }) => <Stack gap={3}>{children}</Stack>;

const Footer = ({ children }) => <Box mt="auto">{children}</Box>;

const Card = Macro({ Cover, Body, Footer })(
  ({ Cover, Body, Footer }, props) => (
    <Flex
      sx={{
        bg: "bg",
        border: "thin",
        borderColor: "muted",
        borderRadius: "base",
        flexDirection: "column",
      }}
      {...props}
    >
      {Cover}
      <Stack height="100%" p={4} gap={4}>
        {Body}
        {Footer}
      </Stack>
    </Flex>
  )
);

export default Card;

but then I'm trying to use it in the app I see the error:
Failed prop type: Invalid child component `[object Object]`. Must be one of: Cover, Body, Footer

I'm wondering if this is somehow related...

Thanks! I haven't had time to look into this library or API recently, but some of the thinking in the mdx-blocks demo I put up, and the theme-ui library itself is more of a newer iteration on the ideas in this library – I'd recommend digging into that to see if it could solve a similar problem to what you're using macro-components for. (I was actually close to archiving this repo a few days ago)
That said, I'd suggest not using the theme-ui pragma everywhere via the Babel plugin and seeing if the issue persists? The other thing to try would be to only include one component at a time to narrow down which one is causing the error to throw, which I think happens if the element.type.macroName property is missing

Yeah, I’m going to spend some time investigating. But since you said it’s kinda deprecated, I’ll probably do something different. Thanks for the quick reply I’ll check out the blocks repo.


Ionicons

URL
https://ionic.io/ionicons/v4
Description of the above provided site
Ionicons is a great website that will give a lot Premium designed icons for use in web, iOS, Android, and desktop apps. Support for SVG and web font. Completely open source, MIT licensed and built by the Ionic Framework team
Category
Frontend
Sub Category
cat = frontend
subcat = Icons

Can I raise a PR for it ?

Sure!

Ok , I will create the PR.


⚠ Jenkins Plugins Artifactory has degraded performance

In e8bc807, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1372 ms

Resolved: Jenkins Plugins Artifactory performance has improved in a72f598 after 41 minutes.


No running component satisfies the requirement

After trying to deploy, it seems like there's an issue with locating a component.
The components on my deployment can be seen as:

Please see below my greengrass.log
2021-11-18T20:27:14.445Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Received new deployment notification. Requesting details. {ThingName=MyJetson}
2021-11-18T20:27:14.542Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Received Iot job description. {Status=QUEUED, queueAt=Thu Nov 18 20:27:13 GMT 2021, ThingName=MyJetson, JobId=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:14.544Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Added the job to the queue. {ThingName=MyJetson, JobId=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:23.297Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Received deployment in the queue. {deploymentId=db6e4746-9ed7-471a-8913-3f05d559faaa, DeploymentType=IOT_JOBS, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:27:23.305Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Received deployment document in queue. {document={"configurationValidationPolicy":{"timeout":60.0},"components":{"aws.greengrass.JetsonDLRImageClassification":{"version":"1.0.0"},"aws.greengrass.Cli":{"version":"2.5.0"},"variant.Jetson.ImageClassification.ModelStore":{"version":"0.1.1"},"variant.Jetson.DLR":{"version":"1.0.0"},"aws.greengrass.LocalDebugConsole":{"version":"2.2.3"},"aws.greengrass.Nucleus":{"version":"2.5.0"}},"componentUpdatePolicy":{"timeout":60.0,"action":"NOTIFY_COMPONENTS"},"schemaDate":"2021-05-17","deploymentId":"990c234a-1f20-40ff-a7ff-1832392abc56","creationTimestamp":1.637267232664E12,"deploymentName":"Deployment for MyJetsonGroup","failureHandlingPolicy":"DO_NOTHING","configurationArn":"arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:8","requiredCapabilities":[]}, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:27:23.348Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.IotJobsHelper: Updating status of persisted deployment. {Status=IN_PROGRESS, StatusDetails={}, ThingName=MyJetson, JobId=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:23.505Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=IN_PROGRESS, ThingName=MyJetson, JobId=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:23.609Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Create work directory for new deployment. {deploymentId=arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:8, link=/greengrass/v2/deployments/ongoing, directory=/greengrass/v2/deployments/arn.aws.greengrass.eu-west-2.576888525644.configuration.thinggroup+MyJetsonGroup.8}
2021-11-18T20:27:23.612Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist deployment metadata. {file=/greengrass/v2/deployments/arn.aws.greengrass.eu-west-2.576888525644.configuration.thinggroup+MyJetsonGroup.8/deployment_metadata.json, deploymentId=arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:8}
2021-11-18T20:27:23.620Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Started deployment execution. {serviceName=DeploymentService, currentState=RUNNING, deployment=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:23.620Z [INFO] (pool-2-thread-19) com.aws.greengrass.deployment.DeploymentService: deployment-task-execution. Starting deployment task. {Deployment service config={ComponentToGroups={aws.greengrass.Nucleus={}}, dependencies=[], GroupToRootComponents={}, runtime={ProcessedDeployments={}}, version=0.0.0}, deploymentId=arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:8, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:27:23.624Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.DependencyResolver: resolve-group-dependencies-start. Start to resolve group dependencies. {targetComponents=[aws.greengrass.JetsonDLRImageClassification, aws.greengrass.Cli, variant.Jetson.ImageClassification.ModelStore, variant.Jetson.DLR, aws.greengrass.LocalDebugConsole, aws.greengrass.Nucleus], componentToVersionRequirements={aws.greengrass.JetsonDLRImageClassification={thinggroup/MyJetsonGroup==1.0.0}, aws.greengrass.Cli={thinggroup/MyJetsonGroup==2.5.0}, variant.Jetson.ImageClassification.ModelStore={thinggroup/MyJetsonGroup==0.1.1}, variant.Jetson.DLR={thinggroup/MyJetsonGroup==1.0.0}, aws.greengrass.LocalDebugConsole={thinggroup/MyJetsonGroup==2.2.3}, aws.greengrass.Nucleus={thinggroup/MyJetsonGroup==2.5.0}}}
2021-11-18T20:27:23.625Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.JetsonDLRImageClassification, versionRequirements={thinggroup/MyJetsonGroup==1.0.0}}
2021-11-18T20:27:23.626Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={thinggroup/MyJetsonGroup==1.0.0}, ComponentName=aws.greengrass.JetsonDLRImageClassification}
2021-11-18T20:27:23.627Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:27:23.628Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=aws.greengrass.JetsonDLRImageClassification-v1.0.0}
2021-11-18T20:27:23.629Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:27:23.851Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:27:23.852Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=aws.greengrass.JetsonDLRImageClassification-v1.0.0}
2021-11-18T20:27:23.860Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=variant.Jetson.ImageClassification.ModelStore, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.2.0, thinggroup/MyJetsonGroup==0.1.1}}
2021-11-18T20:27:23.860Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.2.0, thinggroup/MyJetsonGroup==0.1.1}, ComponentName=variant.Jetson.ImageClassification.ModelStore}
2021-11-18T20:27:23.861Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:27:23.862Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=variant.Jetson.ImageClassification.ModelStore-v0.1.1}
2021-11-18T20:27:23.862Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:27:24.052Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:27:24.053Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=variant.Jetson.ImageClassification.ModelStore-v0.1.1}
2021-11-18T20:27:24.058Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=variant.Jetson.DLR, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.4.0, thinggroup/MyJetsonGroup==1.0.0}}
2021-11-18T20:27:24.058Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.4.0, thinggroup/MyJetsonGroup==1.0.0}, ComponentName=variant.Jetson.DLR}
2021-11-18T20:27:24.059Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:27:24.060Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=variant.Jetson.DLR-v1.0.0}
2021-11-18T20:27:24.060Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:27:24.278Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:27:24.279Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=variant.Jetson.DLR-v1.0.0}
2021-11-18T20:27:24.283Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.Nucleus, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.5.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:27:24.283Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.5.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}, ComponentName=aws.greengrass.Nucleus}
2021-11-18T20:27:24.284Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:27:24.285Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Can't find a local candidate that satisfies the requirement.. {}
2021-11-18T20:27:24.286Z [INFO] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:27:24.461Z [ERROR] (pool-2-thread-20) com.aws.greengrass.componentmanager.ComponentManager: Failed to negotiate version with cloud and no local version to fall back to. {componentName=aws.greengrass.Nucleus, versionRequirement={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.5.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:27:38.622Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Current deployment finished. {DeploymentId=db6e4746-9ed7-471a-8913-3f05d559faaa, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:27:38.705Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.IotJobsHelper: Updating status of persisted deployment. {Status=FAILED, StatusDetails={deployment-failure-cause=Failed to negotiate component 'aws.greengrass.Nucleus' version with cloud and no local applicable version satisfying requirement '{aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.5.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}'., detailed-deployment-status=FAILED_NO_STATE_CHANGE}, ThingName=MyJetson, JobId=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:38.898Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=FAILED, ThingName=MyJetson, JobId=db6e4746-9ed7-471a-8913-3f05d559faaa}
2021-11-18T20:27:39.021Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist link to last deployment. {link=/greengrass/v2/deployments/previous-failure}
2021-11-18T20:27:39.025Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Clean up link to earlier deployment. {link=/greengrass/v2/deployments/previous-failure}
2021-11-18T20:27:39.323Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Received empty jobs in notification . {ThingName=MyJetson}
2021-11-18T20:51:09.709Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Received new deployment notification. Requesting details. {ThingName=MyJetson}
2021-11-18T20:51:09.805Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Received Iot job description. {Status=QUEUED, queueAt=Thu Nov 18 20:51:08 GMT 2021, ThingName=MyJetson, JobId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:09.810Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Added the job to the queue. {ThingName=MyJetson, JobId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:24.107Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Received deployment in the queue. {deploymentId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e, DeploymentType=IOT_JOBS, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:51:24.111Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Received deployment document in queue. {document={"configurationValidationPolicy":{"timeout":60.0},"components":{"aws.greengrass.JetsonDLRImageClassification":{"version":"1.0.0"},"aws.greengrass.Cli":{"version":"2.5.0"},"variant.Jetson.ImageClassification.ModelStore":{"version":"0.1.1"},"variant.Jetson.DLR":{"version":"1.0.0"},"aws.greengrass.LocalDebugConsole":{"version":"2.2.3"},"aws.greengrass.Nucleus":{"version":"2.0.5"}},"componentUpdatePolicy":{"timeout":60.0,"action":"NOTIFY_COMPONENTS"},"schemaDate":"2021-05-17","deploymentId":"d3c42220-2593-4d2d-85f3-965d90cfc887","creationTimestamp":1.637268667598E12,"deploymentName":"Deployment for MyJetsonGroup","failureHandlingPolicy":"DO_NOTHING","configurationArn":"arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:9","requiredCapabilities":[]}, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:51:24.196Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.IotJobsHelper: Updating status of persisted deployment. {Status=IN_PROGRESS, StatusDetails={}, ThingName=MyJetson, JobId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:24.369Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=IN_PROGRESS, ThingName=MyJetson, JobId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:24.478Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Create work directory for new deployment. {deploymentId=arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:9, link=/greengrass/v2/deployments/ongoing, directory=/greengrass/v2/deployments/arn.aws.greengrass.eu-west-2.576888525644.configuration.thinggroup+MyJetsonGroup.9}
2021-11-18T20:51:24.483Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist deployment metadata. {file=/greengrass/v2/deployments/arn.aws.greengrass.eu-west-2.576888525644.configuration.thinggroup+MyJetsonGroup.9/deployment_metadata.json, deploymentId=arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:9}
2021-11-18T20:51:24.497Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Started deployment execution. {serviceName=DeploymentService, currentState=RUNNING, deployment=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:24.500Z [INFO] (pool-2-thread-21) com.aws.greengrass.deployment.DeploymentService: deployment-task-execution. Starting deployment task. {Deployment service config={ComponentToGroups={aws.greengrass.Nucleus={}}, dependencies=[], GroupToRootComponents={}, runtime={ProcessedDeployments={}}, version=0.0.0}, deploymentId=arn:aws:greengrass:eu-west-2:576888525644:configuration:thinggroup/MyJetsonGroup:9, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:51:24.508Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.DependencyResolver: resolve-group-dependencies-start. Start to resolve group dependencies. {targetComponents=[aws.greengrass.JetsonDLRImageClassification, aws.greengrass.Cli, variant.Jetson.ImageClassification.ModelStore, variant.Jetson.DLR, aws.greengrass.LocalDebugConsole, aws.greengrass.Nucleus], componentToVersionRequirements={aws.greengrass.JetsonDLRImageClassification={thinggroup/MyJetsonGroup==1.0.0}, aws.greengrass.Cli={thinggroup/MyJetsonGroup==2.5.0}, variant.Jetson.ImageClassification.ModelStore={thinggroup/MyJetsonGroup==0.1.1}, variant.Jetson.DLR={thinggroup/MyJetsonGroup==1.0.0}, aws.greengrass.LocalDebugConsole={thinggroup/MyJetsonGroup==2.2.3}, aws.greengrass.Nucleus={thinggroup/MyJetsonGroup==2.0.5}}}
2021-11-18T20:51:24.509Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.JetsonDLRImageClassification, versionRequirements={thinggroup/MyJetsonGroup==1.0.0}}
2021-11-18T20:51:24.509Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={thinggroup/MyJetsonGroup==1.0.0}, ComponentName=aws.greengrass.JetsonDLRImageClassification}
2021-11-18T20:51:24.510Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:51:24.511Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=aws.greengrass.JetsonDLRImageClassification-v1.0.0}
2021-11-18T20:51:24.512Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:24.750Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:24.751Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=aws.greengrass.JetsonDLRImageClassification-v1.0.0}
2021-11-18T20:51:24.760Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=variant.Jetson.ImageClassification.ModelStore, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.2.0, thinggroup/MyJetsonGroup==0.1.1}}
2021-11-18T20:51:24.761Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.2.0, thinggroup/MyJetsonGroup==0.1.1}, ComponentName=variant.Jetson.ImageClassification.ModelStore}
2021-11-18T20:51:24.761Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:51:24.762Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=variant.Jetson.ImageClassification.ModelStore-v0.1.1}
2021-11-18T20:51:24.762Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:24.917Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:24.918Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=variant.Jetson.ImageClassification.ModelStore-v0.1.1}
2021-11-18T20:51:24.922Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=variant.Jetson.DLR, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.4.0, thinggroup/MyJetsonGroup==1.0.0}}
2021-11-18T20:51:24.922Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=0.1.1 <1.4.0, thinggroup/MyJetsonGroup==1.0.0}, ComponentName=variant.Jetson.DLR}
2021-11-18T20:51:24.923Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:51:24.924Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=variant.Jetson.DLR-v1.0.0}
2021-11-18T20:51:24.924Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:25.074Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:25.074Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=variant.Jetson.DLR-v1.0.0}
2021-11-18T20:51:25.078Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.Nucleus, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:51:25.080Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}, ComponentName=aws.greengrass.Nucleus}
2021-11-18T20:51:25.081Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found running component which meets the requirement and use it.. {ComponentIdentifier=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.081Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.082Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:25.263Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:25.264Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.270Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.Nucleus, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:51:25.270Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}, ComponentName=aws.greengrass.Nucleus}
2021-11-18T20:51:25.271Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found running component which meets the requirement and use it.. {ComponentIdentifier=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.271Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.272Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:25.448Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:25.448Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.453Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.Nucleus, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:51:25.453Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}, ComponentName=aws.greengrass.Nucleus}
2021-11-18T20:51:25.454Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found running component which meets the requirement and use it.. {ComponentIdentifier=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.455Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.455Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:25.638Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:25.638Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=aws.greengrass.Nucleus-v2.0.5}
2021-11-18T20:51:25.643Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.Cli, versionRequirements={thinggroup/MyJetsonGroup==2.5.0}}
2021-11-18T20:51:25.644Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={thinggroup/MyJetsonGroup==2.5.0}, ComponentName=aws.greengrass.Cli}
2021-11-18T20:51:25.644Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:51:25.645Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Found the best local candidate that satisfies the requirement.. {LocalCandidateId=aws.greengrass.Cli-v2.5.0}
2021-11-18T20:51:25.645Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:25.835Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-end. Negotiated version with cloud. {}
2021-11-18T20:51:25.836Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-end. Resolved component version.. {ResolvedComponent=aws.greengrass.Cli-v2.5.0}
2021-11-18T20:51:25.842Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: resolve-component-version-start. Resolving component version starts. {component=aws.greengrass.Nucleus, versionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, aws.greengrass.Cli=>=2.5.0 <2.6.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:51:25.843Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Searching for best candidate locally on the device.. {VersionRequirements={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, aws.greengrass.Cli=>=2.5.0 <2.6.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}, ComponentName=aws.greengrass.Nucleus}
2021-11-18T20:51:25.844Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: No running component satisfies the requirement. Searching in the local component store.. {}
2021-11-18T20:51:25.844Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Can't find a local candidate that satisfies the requirement.. {}
2021-11-18T20:51:25.845Z [INFO] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: negotiate-version-with-cloud-start. Negotiating version with cloud. {}
2021-11-18T20:51:26.067Z [ERROR] (pool-2-thread-22) com.aws.greengrass.componentmanager.ComponentManager: Failed to negotiate version with cloud and no local version to fall back to. {componentName=aws.greengrass.Nucleus, versionRequirement={aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, aws.greengrass.Cli=>=2.5.0 <2.6.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}}
2021-11-18T20:51:39.501Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentService: Current deployment finished. {DeploymentId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e, serviceName=DeploymentService, currentState=RUNNING}
2021-11-18T20:51:39.586Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.IotJobsHelper: Updating status of persisted deployment. {Status=FAILED, StatusDetails={deployment-failure-cause=Failed to negotiate component 'aws.greengrass.Nucleus' version with cloud and no local applicable version satisfying requirement '{aws.greengrass.JetsonDLRImageClassification=>=2.0.3 <2.1.0, thinggroup/MyJetsonGroup==2.0.5, aws.greengrass.Cli=>=2.5.0 <2.6.0, variant.Jetson.ImageClassification.ModelStore=>=2.0.3 <2.1.0, variant.Jetson.DLR=>=2.0.3 <2.1.0}'., detailed-deployment-status=FAILED_NO_STATE_CHANGE}, ThingName=MyJetson, JobId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:39.780Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=FAILED, ThingName=MyJetson, JobId=9edbb600-30b7-4f26-9fcc-b1d89b062d7e}
2021-11-18T20:51:39.887Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist link to last deployment. {link=/greengrass/v2/deployments/previous-failure}
2021-11-18T20:51:39.891Z [INFO] (pool-2-thread-14) com.aws.greengrass.deployment.DeploymentDirectoryManager: Clean up link to earlier deployment. {link=/greengrass/v2/deployments/previous-failure}
2021-11-18T20:51:41.651Z [INFO] (Thread-3) com.aws.greengrass.deployment.IotJobsHelper: Received empty jobs in notification . {ThingName=MyJetson}

I have solved the issue by removing two components aws.greengrass.Cli and aws.greengrass.LocalDebugConsole


DCKUBE-629: Modified bitbucket elasticsearch example

Bitbucket Elasticsearch example was broken for installing Elasticsearch within Kubernetes cluster. Fixed the command to install a specific Elasticsearch application version. Also added instruction on how to test Elasticsearch connection via Bitbucket UI.

I'd really draw attention to the fact that ES on K8s is 3 instances with hard session affinity, i.e. you'll require 3 worker nodes (that's the default). Also, default cpu/mem requests are 1000m/2Gi, which is also something to be aware of. Finally, K8s cluster needs pre-configured storage and the default claim is 30GB (and since it's pvc template, you;ll need at least 3 volumes)
While we shouldn't support installation of ES Helm chart itself, I think it's good to mention that just helm repo add and helm install may not always work out of the box. Maybe we should mention pre-reqs and reference values.yaml?

@bianchi2 Agree. I've added an info box after Elasticsearch install commands regarding the pre-reqs.

@yzha645 not sure if you're aware of this, but you can render the docs via the browser as you make changes to get an idea of how they will look see https://github.com/atlassian/data-center-helm-charts/blob/main/DEVELOPMENT.md#technical-documentation for details

@yzha645 this needs to be converted to an !!!info box


Electrification of a table with previously dropped columns fails

Trying to electrify a table with at least one column removed via ALTER TABLE ... DROP COLUMN ... fails with an error that looks like this:
=# ALTER TABLE foo ENABLE ELECTRIC;
ERROR:  syntax error at or near "-"
LINE 24:     "__reordered_........pg.dropped.16........" -,
                                                         ^
QUERY:  CREATE TABLE electric.shadow__public__foo (
[...]
    "__reordered_........pg.dropped.16........" -,
    "__reordered_........pg.dropped.17........" -,
[...]
    "_tag_........pg.dropped.16........" electric.tag,
    "_tag_........pg.dropped.17........" electric.tag,
[...]
)
CONTEXT:  PL/pgSQL function electric.ddlx_make_or_update_shadow_tables(text,text,oid) line 63 at EXECUTE
SQL statement "SELECT electric.ddlx_make_or_update_shadow_tables('CREATE TABLE', _schema, _oid)"
PL/pgSQL function electric.electrify(text,text) line 50 at PERFORM
SQL statement "CALL electric.electrify(table_name)"
PL/pgSQL function electric.enable(text) line 3 at CALL

The error is caused by the fact that the electric.lookup_columns() SQL function returns information about columns that have been dropped. Postgres keeps records for such columns in the pg_attribute system catalog but renames them to ............pg.dropped.N........ and sets their type OID to 0.
The solution is to filter out such records when building up a column list in electric.lookup_columns().

Curious, what does the release cadence look like? When is this fix available in GH releases, npm and docker hub?

@divyenduz Correct, it's included in the canary image on Docker Hub as that one is built from the main branch on every push to it.
As for a public release, the next one is coming within the next two weeks.


Not able to create build on Windows platform (Tried using different methods)

Description
I tried exporting jar files using bash code on windows, i even used many work around but it provides same error with json methods and some language exceptions almost 22 errors in row
Steps to Reproduce the Problem
1: Try to compile the whole build, you will receive errors with json and client api (Ref to issue_windows_1)
2: If you build and start client separately as instructed, language exception error will come out may its because wrong path directory to main class - not sure (Ref to issue_windows_2)
Expected Behavior
It should create a build and run
Actual Behavior
Doesn't work, shows error

Platform: Windows 10, MAC OS

Additional info
Well there is a strange thing , it works correctly with linux environment, i dont know if windows is lacking bash support. Looks like we need to work on Linux, if anybody is able to resolve it please comment, i myself looking into it.

I tried this also yeaterday evening, seems that java can't find simple-json.jar. Perhaps classpath should be represented differently in windows?

This like all issues, can be fixed by contributors. Anyone catching this? I can say that yes, the build.sh are for *nix (linux, mac) and require some editing for windows. Not complex, little googling or a peek to stackoverflow.com helps. So anyone can create a build file for Windows, e.g. a .bat file. Just a side note, I am not a contributor here ;)

Yes i have bat file workaround in my mind, i will do that in evening. but strange thing is it is happening with Mac as well, my group has mac machine it was almost on it as well.

You need to build the library first and it must be exported to a .jar file in the correct directory (build.sh does this). The json lib must also reside there. Then you can build the console, client and server. Maybe this is the problem.


Internal: Fix test_json.py from @facelessuser

https://github.com/divmain/GitSavvy/commit/4bc4bf32636df4a5f620193bbffe34686ec76bf9

The maintainers of this repo require that all pull request submitters agree and adhere to the following:

[x] I have read and agree to the contribution guidelines.
(required)
[x] All related documentation has been updated to reflect the changes made. (required)
[x] My commit messages are cleaned up and ready to merge. (required)

Should we report these two maintainerd failures as bugs?

semantic version test cannot be cancelled by setting semver to false
msg test for lines shorter than 72 seems broken (at least for some cases).
Typo in Commit message lines much be shorter where much => must`.

Though, I'm not sure @divmain has time to maintain maintainerd, there are still some PRs pending from september.


[$125] [Search v2.1] Update card filter to include the last four digits to the options

Coming from here, we should include the last four digits of the card in the options so that we know which card we're selecting when there are multiple options with the same name
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~012701341a68cca511
Upwork Job ID:<PHONE_NUMBER>220699994
Last Price Increase: 2024-08-14

Issue OwnerCurrent Issue Owner: @

@Expensify/design do we have a mock for how we wanna show the last 4 digits of the card?

I don't mind how it's shown in this proposal video, though since the user is likely looking for a specific card, I wonder if we should make it more prominent in the title.
@Expensify/design what do you all think?

[!NOTE]
I grabbed a list with cards from Jon's card feed Figma file, if these icons and bank/card names are wrong, please forgive me. Just trying to get a sense of the type styles.

Also, will we have icons that match the cards? Or will we need something generic to sit on that left side?

Dig that mock though I kinda feel like we should have "Ending in 1414" or something. Might be repetative, but I feel that just the numbers below the card name feels a tad odd. Maybe even "···· 3128" or something

Big fan of reusing the same card rows from the card feed project, and I think Jon's ideas for making the card number slightly more obvious are great too.

though I kinda feel like we should have "Ending in 1414" or something.

Yeah that makes perfect sense to me. I like Ending in #### because that's the way I typically see card numbers represented out in the wild wild web.

Dig that mock
Big fan of reusing the same card rows from the card feed project

So which style from here are you saying you all prefer? The left or the right? With the addition of the Ending in text, I think I prefer the right. Somethin' like this:

Alts just for fun:

Nice! Yeah, the first mock you just shared is what I had in mind. I could get down with either of the alts too though. I guess you could argue the first mock is more consistent with existing selection list options though? Where we always use the subheading/description text on a second line?

Totally agree with you. I think the first option is the way to go!

Definitely agree that the first mock is the right fit! 👏 GG

What's the latest on this one? Is a contributor working on this or is it internal @luacmartins @ikevin127?

I think we're waiting on an updated proposal with the latest mocks

@daledah's updated proposal looks good to me. The main solution looks good for this one based on design's team feedback (first picture from https://github.com/Expensify/App/issues/47324#issuecomment-2291285295).
🎀👀🎀 C+ reviewed

❓ @Expensify/design I was reviewing the PR and noticed that the author did not add translations to the Ending in text yet. I checked out our current translations for similar / existing ones and I found this one which is not used anywhere:
cardEnding: ({cardNumber}) => `Card ending in ${cardNumber}`,

Question is: Can we use Card ending in 5555 or should we stick with the first mock from https://github.com/Expensify/App/issues/47324#issuecomment-2291285295 and add new & translate Ending in ?

We want to use the new "Ending in..." and we should do that everywhere.

⚠ We just got confirmation on Slack that the Deploy Checklist: New Expensify 2024-08-26 which includes the PR of this issue was only deployed to production today in Deploy Checklist: New Expensify 2024-08-28. More context on why this happened can be found in this Slack thread and this Slack comment.
Given the context above, this issue should be on [HOLD for Payment 2024-09-6] according to today’s production deploy from Deploy Checklist: New Expensify 2024-08-28.
cc @lschurr @luacmartins

The PR solving this issue was deployed to prod in v9.0.26-6

This is not overdue. Payment is Friday

Payment summary:

Contributor: @daledah (can you provide your Upwork profile here or apply to the job?)
Reviewer: @ikevin127 $125 (paid in Upwork)

Bump on this @daledah

Thanks! Offer sent @daledah https://www.upwork.com/nx/wm/offer/103900230


Topicref to bookmap breaks PDF generation

In a regular DITA Map you have a bookmap referenced like this:
<map>
    <title>bla</title>
   <topicref href="first.dita">
       <topicref href="bookmap.ditamap" format="ditamap"/>
   </topicref>

The PDF processing breaks with:
        Caused by: org.apache.fop.fo.ValidationException: "fo:page-sequence" is not a valid child of "fo:block"! (See position 3:91)
        	at org.apache.fop.events.ValidationExceptionFactory.createException(ValidationExceptionFactory.java:38)
        	at org.apache.fop.events.EventExceptionManager.throwException(EventExceptionManager.java:58)
        	at org.apache.fop.events.DefaultEventBroadcaster$1.invoke(DefaultEventBroadcaster.java:175)
        	at com.sun.proxy.$Proxy5.invalidChild(Unknown Source)
        	at org.apache.fop.fo.FONode.invalidChildError(FONode.java:561)
        	at org.apache.fop.fo.FONode.invalidChildError(FONode.java:537)
        	at org.apache.fop.fo.flow.Block.validateChildNode(Block.java:293)
        	at org.apache.fop.fo.FOTreeBuilder$MainFOHandler.startElement(FOTreeBuilder.java:276)
        	at org.apache.fop.fo.FOTreeBuilder.startElement(FOTreeBuilder.java:179)
        	at net.sf.saxon.event.ContentHandlerProxy.startContent(ContentHandlerProxy.java:398)
        	... 66 more

topicrefBookmap.zip

Probably most of the XSLT templates from "org.dita.pdf2\xsl\fo\commons.xsl" which process bookmap references like "processTopicChapter" should not start a "page-sequence" if they are not on the first level in the resolved map.

Reproduced in 3.0, and thinking that the suggestion above is probably a good one.

This proved more troublesome than I thought but I have a fix up that can be tested.
It keeps current processing the same for bookmap (no moving around of page sequences, no renaming of templates -- only breaking existing ones down a bit -- so any overrides will still work).
The fix allows you to use map references like the one submitted above, where:

Referencing map is not a bookmap
The reference is made within the context of another topic (so bookmap ends up nested in a topic)
The bookmap uses a chapter, part, preface / front matter topic, appendix, or appendices (each of these would previously set up a new page sequence inside the existing fo:block)


Removes apostrophe before year in commitment

There is currently a single quotation mark/apostrophe before the year in commitments (e.g., Oct 1, '2021). This removes that (e.g., Oct 1, 2021).

As I'm looking, the other fix could probably be removing the semicolon after the year when a commitment doesn't have an end date inside the range shown...

Thank you!


[docs] Mention that you can run Studio tests

https://twitter.com/karloabapo/status/1103522987167236096

Fixed in https://github.com/DevExpress/testcafe/pull/3690


Wildcards don't work until you Refresh wildcards in Wildcards manager

Stumbled upon this problem and was looking for a solution. Not sure if it's bug or not, but Readme does not mention this in the Troubleshooting section.
If you've made wildcard txt files outside the web ui, and you're getting the "No values found for wildcard some/wildcard" you need to go to the Wildcards editor and press Refresh wildcards button.
It would be better to make it automatic before image generation.

See https://github.com/adieyal/sd-dynamic-prompts/pull/386.


Add data retention support for metric views based on continuous aggregates

Continuous aggregates rely on hypertables to store their aggregated data. This
change tries to check for metric views which are based on them to apply the same
data retention system as for raw metric data. Users will be able to set data
retention rules for them same as for other raw metrics.

bors r+


Update codeql-analysis.yml

Tackling codeQL startup failure error

Is this any issue? Thia is not needed to be updated currently.


Possible Thread Leak in ...

Description
The number of threads increases with each transaction in some cases (hard to replicate exactly). This is an example thread's stack trace, all of the unexpected and additional threads have the same trace:
["\n# ThreadID:<PHONE_NUMBER>48768","File: \"/usr/lib/python3.8/threading.py\", line 890, in _bootstrap"," self._bootstrap_inner()","File: \"/usr/lib/python3.8/threading.py\", line 932, in _bootstrap_inner"," self.run()","File: \"/usr/lib/python3.8/threading.py\", line 870, in run"," self._target(*self._args, **self._kwargs)","File: \"/home/ubuntu/adam-action-graph-service/action_selection/venv/lib/python3.8/site-packages/typedb/stream/request_transmitter.py\", line 108, in run"," self._permission_to_run.acquire()","File: \"/usr/lib/python3.8/threading.py\", line 433, in acquire"," self._cond.wait(timeout)","File: \"/usr/lib/python3.8/threading.py\", line 302, in wait"," waiter.acquire()"]
Environment

Ubuntu
TypeDB 2.5.0
TypeDB client-python 2.5.0
Python version 3.8

Reproducible Steps
Unclear at the moment.
Expected Output
"Request transmitter" threads to be closed at the end of a transaction.
Actual Output
The number of threads grows with each transaction, even once closed.

It looks like this issue is now resolved, thank you.


File upload example

Is there an example available how to implement a file upload including schema and handling in the backend?

https://doctrine-orm-graphql.apiskeletons.dev/en/latest/types.html


Add apt-get update to Linux CI build

Add an "apt-get update" command for the Linux builds defined in the ci_build.yml and release_build.yml actions files.

CI gfxreconstruct build queued with queue ID 13550.

CI gfxreconstruct build # 76 running.

CI gfxreconstruct build # 76 passed.


Noob question: How to get value of sim result (voltage in electrostatic sim) for a surface/element with known GMesh attr. ID

Hello,
I am using electromagnetic miniapp , specifically volta.cpp and used it to do an electrostatic sim. My mesh was generated with GMesh and sim was run successfully. I can load result and visualize it using GLVis or Paraview (.pvu generated also).
All I need now is to extract values (voltages) on a particular surface with a known gmesh's attribute ID, preferably from data structures within the C++ sim. I don't have FE background and deciphering C++ library is not straightforward when I tried.
Any example or hint that you could point me to would be really appreciated.
Regards.

Hello, @aalavi,
This is a good question but its answer is perhaps more complicated than it would appear.
If you need voltages at a fixed number of locations which you know in advance then you can extract these voltages using code similar to that found in get-values.  This tool does not use the boundary attributes it simply searches the mesh for the elements containing the coordinates you provide. Clearly this can be expensive on a large mesh.
The more complicated answer involves looking at the problem as a finite element field. In the volta mini app the voltage is a continuous field so its precise value at any given point may not be that important. Certainly point-wise values will change as you refine your mesh and the solution becomes more accurate but such values will also be different on different meshes which use the same mesh spacing.  Perhaps the average voltage over some region would be more useful.
To compute the average voltage on a surface you can use a LinearForm object to integrate your solution field over the surface. You can use the same LinearForm to compute the area of the surface. The ratio of the two would then be the average voltage. Something very similar is done, in a more complicated manner, in ex27p.cpp.
I can probably give more details on either approach if these sound useful.
Best wishes,
Mark

Thank you Mark for your response!
The first approach I guess might suffice in my case. I have a floating (not connected to a voltage source) surface (sensing metal electrode) with Neumann boundary condition imposed on it in sim on which I would like to get the induced voltage on. It is metal so the whole surface should have identical voltage readout (or similar).
Simplified example would be a two-plate capacitor model with plates connected to voltages 0 and 100V, and I have my sense small metal plate somewhere to sense the induced voltage on it.
Would this example possibly help you elaborate a bit on proper method that I could pursue? I am going to study both approaches regardless out of curiosity.  I will study LinearForm structure, and try to understand ex27p as well.
Thank you in advance for helping out.
Regards


[FLINK-1481] Fixes flakey JobManagerITCase

The sometimes failing sender tasks are now guaranteed to have at least one task which fails.

Good one, thank you!
+1 to merge

I'll merge it.