[Q] How to use wandb.init to change project settings?

In the docs: https://docs.wandb.ai/guides/app/pages/project-page#project-defaults, it says

These settings can also be specified by passing arguments to wandb.init.

But how? Specifically, how to set project privacy through wandb.init?

Hi @why-in-Shanghaitech thanks for reporting this, I will create a documentation update request as project permissions settings don't seem to be an option from the wandb.init() call.
However, you could use the API to create a project before creating a new run in it as follows:
import wandb

api = wandb.Api()
api.client.execute(
    api.CREATE_PROJECT,
    {
        'entityName': 'your-entity-name',
        'name': 'your-project-name',
        'access': 'PRIVATE',
    },
)

You could have access set to USER_READ (for Public project) or USER_WRITE (for Open project). I hope this helps, please let me know if that won't work  for you.

Thank you very much for the reply! Unfortunately, it does not work in version 0.12.9, since it raises AttributeError: 'Api' object has no attribute 'CREATE_PROJECT'. Though it works in the latest version 0.14.0, I cannot log results in an environment without the Internet (even with environment variable WANDB_MODE as dryrun and setting mode="offline" in wandb.init). Version 0.14.0 will try to access localhost, which is impossible in my case.
   6    File "<path>/wandb/sdk/lib/sock_client.py", line 102, in connect
   7:     s.connect(("localhost", port))
   8  ConnectionRefusedError: [Errno 111] Connection refused

Since this is a new problem, I'm closing this issue. I have no plan to create a new issue because currently I'll just stick to version 0.12.9.

Hi @why-in-Shanghaitech thanks for the confirmation that this worked for you on v0.14.0.
I agree that the new issue would be best to go in a new thread to be tracked but since you've mentioned you aren't planning on making one, I was wondering if we could quickly troubleshoot this one, are you running Sweeps or simple runs? The dryrun has been deprecated in our most recent versions, so you will need to either set it from the env WANDB_MODE=offline, or from wandb.init(.., mode="offline"). Do you have any other WANDB_ environment variables set, and are you logging runs to our public cloud instance wandb.ai or to a Local W&B instance?

Thanks for the reply! I try to change the environment variable, but it still doesn't work. I'll attach the full error message below:
Traceback (most recent call last):
  File "<path>/wandb/sdk/wandb_manager.py", line 117, in _service_connect
    svc_iface._svc_connect(port=port)
  File "<path>/wandb/sdk/service/service_sock.py", line 30, in _svc_connect
    self._sock_client.connect(port=port)
  File "<path>/wandb/sdk/lib/sock_client.py", line 102, in connect
    s.connect(("localhost", port))
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<path>/wandb/sdk/wandb_init.py", line 1140, in init
    wi.setup(kwargs)
  File "<path>/wandb/sdk/wandb_init.py", line 171, in setup
    self._wl = wandb_setup.setup(settings=setup_settings)
  File "<path>/wandb/sdk/wandb_setup.py", line 327, in setup
    ret = _setup(settings=settings)
  File "<path>/wandb/sdk/wandb_setup.py", line 320, in _setup
    wl = _WandbSetup(settings=settings)
  File "<path>/wandb/sdk/wandb_setup.py", line 303, in __init__
    _WandbSetup._instance = _WandbSetup__WandbSetup(settings=settings, pid=pid)
  File "<path>/wandb/sdk/wandb_setup.py", line 114, in __init__
    self._setup()
  File "<path>/wandb/sdk/wandb_setup.py", line 250, in _setup
    self._setup_manager()
  File "<path>/wandb/sdk/wandb_setup.py", line 277, in _setup_manager
    self._manager = wandb_manager._Manager(settings=self._settings)
  File "<path>/wandb/sdk/wandb_manager.py", line 164, in __init__
    sentry_reraise(e, delay=True)
  File "<path>/wandb/util.py", line 214, in sentry_reraise
    raise exc.with_traceback(sys.exc_info()[2])
  File "<path>/wandb/sdk/wandb_manager.py", line 162, in __init__
    self._service_connect()
  File "<path>/wandb/sdk/wandb_manager.py", line 126, in _service_connect
    raise ManagerConnectionRefusedError(message)
wandb.sdk.wandb_manager.ManagerConnectionRefusedError: Connection to wandb service failed since the process is not available.
wandb: ERROR Abnormal program exit
Traceback (most recent call last):
  File "<path>/wandb/sdk/wandb_manager.py", line 117, in _service_connect
    svc_iface._svc_connect(port=port)
  File "<path>/wandb/sdk/service/service_sock.py", line 30, in _svc_connect
    self._sock_client.connect(port=port)
  File "<path>/wandb/sdk/lib/sock_client.py", line 102, in connect
    s.connect(("localhost", port))
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<path>/wandb/sdk/wandb_init.py", line 1140, in init
    wi.setup(kwargs)
  File "<path>/wandb/sdk/wandb_init.py", line 171, in setup
    self._wl = wandb_setup.setup(settings=setup_settings)
  File "<path>/wandb/sdk/wandb_setup.py", line 327, in setup
    ret = _setup(settings=settings)
  File "<path>/wandb/sdk/wandb_setup.py", line 320, in _setup
    wl = _WandbSetup(settings=settings)
  File "<path>/wandb/sdk/wandb_setup.py", line 303, in __init__
    _WandbSetup._instance = _WandbSetup__WandbSetup(settings=settings, pid=pid)
  File "<path>/wandb/sdk/wandb_setup.py", line 114, in __init__
    self._setup()
  File "<path>/wandb/sdk/wandb_setup.py", line 250, in _setup
    self._setup_manager()
  File "<path>/wandb/sdk/wandb_setup.py", line 277, in _setup_manager
    self._manager = wandb_manager._Manager(settings=self._settings)
  File "<path>/wandb/sdk/wandb_manager.py", line 164, in __init__
    sentry_reraise(e, delay=True)
  File "<path>/wandb/util.py", line 214, in sentry_reraise
    raise exc.with_traceback(sys.exc_info()[2])
  File "<path>/wandb/sdk/wandb_manager.py", line 162, in __init__
    self._service_connect()
  File "<path>/wandb/sdk/wandb_manager.py", line 126, in _service_connect
    raise ManagerConnectionRefusedError(message)
wandb.sdk.wandb_manager.ManagerConnectionRefusedError: Connection to wandb service failed since the process is not available.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<path>/original-test/example/model.py", line 43, in <module>
    optimized_params = nni.get_next_parameter()
  File "<path>/nni/nni/trial.py", line 64, in get_next_parameter
    _params = platform.get_next_parameter()
  File "<path>/nni/nni/runtime/platform/local.py", line 60, in get_next_parameter
    wandb_init(params)
  File "<path>/nni/nni/runtime/platform/local.py", line 94, in wandb_init
    wandb.init(
  File "<path>/wandb/sdk/wandb_init.py", line 1181, in init
    raise Exception("problem") from error_seen
Exception: problem

All environment variables starts with WANDB_ are: WANDB_MODE with value offline.
The code for initialization:

wandb.init(
        dir="<path>",

        # set the wandb project where this run will be logged
        project="<random string>",
        name="<random string>",
        id="<random string>",

        # track hyperparameters and run metadata
        config=params,
        anonymous="allow",
        mode="offline"
    )

I am logging to public cloud instance wandb.ai. I am using an anonymous account.

wandb: Currently logged in as: anony-moose-529863. Use `wandb login --relogin` to force relogin

Hi @why-in-Shanghaitech I was wondering if you could try to disable wandb service, if that would help to overcome this issue. Could you please add the following lines before your wandb.init call?
import wandb
import os

os.environ['WANDB_DISABLE_SERVICE']='True'

Hey @why-in-Shanghaitech just checking in here to see if you've tried disabling service, and if that worked for you? thanks!

Sorry for the late reply. I have to say that this issue is more complicated than I thought.
I was developing a patch for NNI with slurm: https://github.com/whyNLP/nni-slurm. The network issue only appears when I use wandb in NNI trials (submitting to slurm). If I submit a job directly to slurm, setting WANDB_MODE='offline' or init(mode='offline') is enough to make the script work.
So the good news is that this is not a common issue for most users. I haven't tried to disable wandb service on NNI. I'm quite busy with my coursework and research these days, but once I get time I'll give it a try and post the results here.

Thanks @why-in-Shanghaitech for developing and sharing this patch, this is great! You may also find useful this unofficial repo regarding wandb on Slurm. Since the original issue has been resolved, I will close this ticket for now. However, please feel free to reopen this or create a new thread once you're available to resume to this and we will be glad to keep investigating!

I had the same error message

wandb.sdk.wandb_manager.ManagerConnectionRefusedError: Connection to wandb service failed since the process is not available.

when using SLURM and Weights and Biases hyper-parameter sweep in parallel.
This error occurs because there is a W&B bug that tries to connect to localhost instead of the W&B controller that schedules which hyper-parameter configuration to run next.
I set os.environ['WANDB_DISABLE_SERVICE']= "True" which completely fixed the problem. This was a difficult bug to solve so please fix it in the next W&B release! Thanks.

@gil2rok Not sure how you discovered that was the solution, but I ran into the same problem when using W&B on Slurm. Thanks for the temporary fix!

@gil2rok sorry for the bug and thanks a lot for finding the source of the issue. We would like to fix this issue for you so you don't need to disable the wandb service, do you think you would be able to provide us with a small reproduction, so we could fix it (allowing both wandb service and proper usage of sweeps?)

@michaelmior are you also using it in combination with sweeps or do you see this error just when using wandb on Slurm?

@kptkin Only in combination with sweeps. I've been able to use wandb logging in other Slrum jobs without issue.

@kptkin Only in combination with sweeps. I've been able to use wandb logging in other Slrum jobs without issue. Could you clarify what WANDB_DISABLE_SERVICE does? It doesn't seem to be documented anywhere.

Thanks for the response.
Sure, it is just some internal implementation details, but high level speaking:

In our default mode of operation we spin up a separate process (which we call service), this process manages all the active runs in your system and responsible for the network and file system operations.
When you disable this flag we revert to a different mode of operation where each run will have it's own process to manage the run. You also lose features like sharing the run across multiple processes and in general, risking a hang if the multiprocessing with other code is not done properly.
For some additional context see here: https://docs.wandb.ai/guides/track/log/distributed-training#enable-wb-service (as of 0.16.0 it is enabled by default, need to update the documentation)

Hope this helps. @michaelmior if you could provide a small reproduction example i would be happy to resolve this issue, so you don't have to disable this service.

@kptkin I've been able to reproduce this using litgpt with the following sweep
method: bayes
metric:
  goal: minimize
  name: val_loss
parameters:
  train.learning_rate:
    min: 0.00001
    max: 0.01
  train.min_lr:
    min: 0.000001
    max: 0.00006
  train.epochs:
    min: 1
    max: 5
  train.weight_decay:
    min: 0.0
    max: 0.05
command:
  - ${env}
  - sbatch
  - --wait
  - ./run.sh
  - --config
  - lora.yaml
  - ${args}

I'm using a custom dataset, but I think using any config like this one will work.
My run.sh script is just a wrapper that sets up my environment. Ultimately it just executes litgpt finetune lora with all the arguments passed in.

I have also run into this issue when running on a server (not using SLURM). os.environ['WANDB_DISABLE_SERVICE']= "True" seems to have fixed it. It was working fine last week and then randomly started throwing the same error as listed previously.

I have similar problems. Iam trying resume a wandb run on a SLURM job, by running:
run = wandb.init(
        entity=entity,
        project=project,
        id=<ide_of_run>,
        resume="must",
    )

And I get the errors:
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
wandb.sdk.wandb_manager.ManagerConnectionRefusedError: Connection to wandb service failed since the process is not available.

I pass my API key via:
export WANDB_API_KEY= in the sbatch script

@DavidoF3 i think your issue is a bit different, could be that your environment is in a bad state and we can't connect to the service process. Could  you make sure that you don't have any value set for "WANDB_SERVICE" env variable? also if you happen to have: ps -ef | grep wandb-service process could you please kill these as well?

@amacrutherford if it throws this error it means you didn't actually disable service as it still tries to connect to the service process. see my comment here: https://github.com/wandb/wandb/issues/5272#issuecomment-2043405987

@kptkin I am able to run a job successfully when the service is disabled. However, I would like to run my job with the service ON because I would like to keep track of training results from the wandb UI.

@kptkin You should be able to run the job with the service disabled and still track the results from the UI. At least, I am able to do so just fine in my jobs.

@michaelmior @DavidoF3 yes service shouldn't affect your ability to track the results in the UI. It is just enable more advanced functionality to manage multiprocessing that log to the same run and other improvements

@kptkin I've been able to reproduce this using litgpt with the following sweep
method: bayes
metric:
  goal: minimize
  name: val_loss
parameters:
  train.learning_rate:
    min: 0.00001
    max: 0.01
  train.min_lr:
    min: 0.000001
    max: 0.00006
  train.epochs:
    min: 1
    max: 5
  train.weight_decay:
    min: 0.0
    max: 0.05
command:
  - ${env}
  - sbatch
  - --wait
  - ./run.sh
  - --config
  - lora.yaml
  - ${args}

I'm using a custom dataset, but I think using any config like this one will work.
My run.sh script is just a wrapper that sets up my environment. Ultimately it just executes litgpt finetune lora with all the arguments passed in.

Thanks for the reproduction, i'm adding it to our log of tasks, hopefully we can solve it for you guys soon!

@michaelmior you are right. The reason why it wasn't updating the UI is because I was running both:
export WANDB_MODE='offline'     -> (suggested in other issues)
export WANDB_DISABLE_SERVICE='True'

If you want to track results in the UI, you just need to run:
export WANDB_DISABLE_SERVICE='True'


IMA HTML5 SDK Doesn't work on MS Edge :)

Well, this includes the demo of the stand alone SDK and the VideoJS Demo.
If I test with tag with non-linear ad, it works again.
Even the companion ad is loaded, but no pre-rolls, mid-rolls, post-rolls :)
http://googleads.github.io/googleads-ima-html5/advanced/

From @cycleos - "the problem appears to be applicable to IE 11 as well, but only on Windows 10.
Tests with IE11 on Windows 7 were fine."

More from @cycleos -  I created two screencaptures to demonstrate the problem itself.
I am opening the Video JS IMA demo on github.io. I am choosing the sample ad tag. I am clicking play. No ads are displayed at all.
I am opening the IMA Stand Alone demo on github.io. I am choosing the sample ad tag. I am clicking play, the companion ad loads, but not the pre-roll.
Hope it helps.
Screencapture for VideoJS IMA:
https://www.youtube.com/watch?v=Wk8Jm0V4Y8w
Screencapture for IMA Stand Alone:
https://www.youtube.com/watch?v=5Jtc0r30FIo
This is IE11 on Windows 10. Same happens on Edge. There are no plugins, no adblockers etc. There are no errors in the console.

Looking into this issue a bit more, it looks like the simple sample[1] works but the advanced sample[2] doesn't. So this is probably an implementation issue within the samples and the video.js plugin, and not an issue within the IMA SDK itself. We're not getting any errors in the Edge console, though. I'll continue to investigate.

At this point it appears to be a creative/media file issue. If I take the tag from the simple sample and put it in the advanced sample it plays fine. Following up with the rest of the IMA team to determine how to fix.

Hi @shawnbuso - Any progress on this? I can't get it to work on Edge or IE11 using Windows 10, following your advice to use simple sample tag and put it in the advanced sample. I am also using VideoJS 5.11 and latest version of  videojs-contrib-ads plugin.

This is still an issue in the IMA SDK so we don't have an update yet unfortunately - once it's fixed in the IMA SDK it will be automatically fixed in the video.js plugin.

Is it possible to somehow force flash for IE and Edge? Single VPAID 1.0 Linear Flash (VAST Inline) from IMA Sample Tags works.

Do you have one tag serving multiple formats, or different tags for different formats? In either case you should be able to add some logic based on the UserAgent of the current browser. If you have different tags, just change the tag passed into the plugin. If it's the same tag, you can pass in mimeTypes to adsRenderingSettings like so:
{
  id: '...',
  adtagUrl: '...',
  adsRenderingSettings: {
    mimeTypes: ['application/x-shockwave-flash']
  }
}

I am working on a SaaS project, where users provide the different tags and our service provides embeddable player and streaming. I will probably need some kind of validation of the tags to check if flash format is supported. Thanks for the quick response :)

This has been resolved by a patch to our ad server and should require no update on your part to see the fix - I was able to successfully play the advanced sample ad on IE11.


Various issues with icon field

[x] after selecting an icon and reopening the icon chooser the selected icon/color is not correctly shown
[x] if jumping to next page before selecting a category then the icon list of the category is empty (pagination should probably be reset)
[ ] the icon object structure contains a flatten array

[x] Font awesome 6 has been released but we refer to a metadata file coming from the master branch of the github project, we should probably upgrade to FA v6 or refer to the v5 branch

Can't reproduce issue with flatten array, will reopen if required


Uninstall events are not being recorded

Hello, as the title says, uninstall events are not being recorded by the sdk.
Package version:
"infobip-mobile-messaging-react-native-plugin": "8.4"
SDK Initialization on index.js:
mobileMessaging.supportedEvents.forEach(event => {
    mobileMessaging.subscribe(event, handleMobileMessagingEvent);
});

mobileMessaging.init(
    {
        applicationCode: app_settings.infobip.appCode,
        ios: { notificationTypes: ['alert', 'badge', 'sound'] },
    },
    () => {},
    error => {
        console.log('MobileMessaging error: ', error?.description);
    }
);
...
AppRegistry.registerRunnable(appName, props => {
    AppRegistry.registerComponent(appName, () => App);
    TrackPlayer.registerPlaybackService(() => Player);
    AppRegistry.runApplication(appName, props);
});

After user login:
mobileMessaging.personalize(
            {
                userIdentity: {
                    emails: [email],
                    externalUserId: id
                },
                userAttributes: {
                    firstName: first_name,
                    lastName: last_name,
                    customAttributes,
                },
                forceDepersonalize: true,
            },
            user => console.log('infobip user personalized', { user }),
            error => console.error('infobip user personalized error', { error })
);

I followed carefully all the required setup steps for Android and IOS, including the addition of the Notification Service Extension on IOS.
Expected behavior:
Users who did uninstall our App must send an "Uninstall" event to the Infobip servers.
Actual behavior:
The sdk is not sending the "Uninstall" event to the infobip servers, so, the installation on th ePerson profile in the Infobip portal stays as qualified to receive push notifications and are not being filtered when making broadcasts, resulting in terrible push notifications campaign results.
Metrics on Infobip portal of user uninstalls since our last release

Uninstall metrics from Firebase for the same date range

Hello @jorgerojas26 ,
We are tracking uninstalls only after confirming that app had been uninstalled by sending push notification to the installation - as described in this article  , it will be removed from People service in up to 48 hours.
We will check your second request more thoroughly: new app version should not cause new registrations to be created in the first place.

Hello @alboldy-ib, thanks for your response.
The thing is, as you can see in the image i posted with the infobip metrics, uninstall events are basically 0 while in Firebase and Google Play Console, for the same time frame, uninstalls events are around 1k.
Yesterday i sent a silent push notification to around 23k users to check how many installations could receive it, here are the metrics from /push/1/statistics API.

As you can see, almost 15k installations throws registration error, which in fact is the 8001 error (see https://www.infobip.com/docs/essentials/response-status-and-error-codes).
How can we resolve this issue? We definitely need to clean up our user base because infobip is not cleaning it.

Hello @jorgerojas26 ,
After receiving error on either trying to deliver push or performing dry-run, installation is marked as to be deleted. Such installation will be removed in up to 48 hours, the logic is not to remove them immediately.

Closing the issue because no additional questions or comments added for a long time.


try catch clause isn't supposed to match more than single statement

see: https://github.com/oceanprotocol/ocean.js/blob/0b50db1d3bf1fa4b7721effe1b6f046732c55409/src/ocean/Assets.ts#L558
The usefulness of a try catch statement, with regard to its capability of matching errors and then addressing them accordingly, comes from matching for one statement many errors.
However, as it's being done in the above example, there are numerous failable statements in a single try catch clause which makes catching particular errors impossible.

Fixed with: https://github.com/oceanprotocol/ocean.js/pull/1524.
The refactoring is not merged yet in the main branch as it is still under testing.


Subcommands or flags?

Should we use subcommands or flags more often?
So:
$ reliably discover --live --kubeconfig

or
$ reliably discover live --config

I think this one ties into #71
I believe if we're defining significant actions or functionality, then that functionality should likely be associated with its own command or sub command to isolate the logic.

I think --live is more of a mode of how to run the command, rather than a real resource on its own
Currently we have discover as a single command on its own, it's neither a resource nor a verb of the cli


Validation on project creation

Should there be more validation on the project form? Currently the only required field is title and project leader email.

a minimum length on project description?
"Department" being a required field
Project Leader's name being a required field
Project Leader's position being a required field

Also the presence of start and end date are validated.

Given this would affect Cardiff and Bangor as well, we should consult with Aaron before implementing this.

@tystakartografen - do you have any thoughts on this one?


Compilations don't show as "Various Artists"

They currently show one of the featured artists.

Good catch, thanks! Gonna fix it in v1.4.0


most of the browser containers can't be stopped

Error response from daemon: driver "aufs" failed to remove root filesystem for
ubuntu@ip-172-30-0-174:~/tools/qps-infra$ docker --version
Docker version 17.09.0-ce, build afdb6d4

issue is fixed by upgrading to latest docker version:
docker --version
Docker version 19.03.8, build afacb8b7f0

verified on new demo env successfully

will be re-verified by Vadim.


Help request: Multiple lines connection point overlapping

Hi @anseki, first of all, congrats for the lib, it's really cool!
I'm dealing with this kind of graphs, see the lines starting from box 3:

I'd need these lines (could be several from/to each box) to not overlap, something like this:

Is there an easy way without manually recalculating the side (and coordenates) to accomplish this?
I was thinking something like using pointAnchor and recalculating an offset amount (based on external information about the lines connecting to the object) on the automatically chosen side before rendering the line, but couldn't find any events triggered.

Hi @maxidirienzo, thank you for the comment.
Sorry, my English is poor.
Do you mean that you want to connect the line to specific point instead of startSocket/endSocket, and the pointAnchor didn't work for that?
If so, could you explain more about the problem of pointAnchor?

pointAnchor is working perfectly, I'm having problems to retrieve a line's pointAnchor after setting it, eg:
const line = new LeaderLine(
 LeaderLine.pointAnchor( element1, {x: '100%', y: 30} ),
 LeaderLine.pointAnchor( element12, {x: '100%', y: (target.ix * 24) + 12} ),

if later I call line.end it does not return the pointAnchor but a function... it's not big deal right now but it would be cool the get the anchor information.

Sorry, what is the "anchor information"?

For example: you initially set the start to LeaderLine.pointAnchor( element1, {x: '100%', y: 30} )
Later you get line.end and there seems to be no way to change the x and y properties of the pointAnchor, instead you need to reassign a new LeaderLine.pointAnchor( element1, {x: '100%', y: 50} ) instance to move the line in the y axis

You can get the pointAnchor instance via line.start (not line.end) because you have set the start.
See document: https://anseki.github.io/leader-line/#pointanchor
You can't change the properties of the instance as you said. That is not required.
A smart way is that saving the data (anchor information) into variables because the data was made by your code (not the library).
For example:
const offset1StartX = '100%',
  offset1StartY = 30;

line.start = LeaderLine.pointAnchor(element1, {x: offset1StartX, y: offset1StartY});

Correct, I got it's the start anchor I need to adjust in order to match my example, but for illustration purposes I copied code that dealt with end.
I've end up doing what you've said when recalculating the line anchor positions:
line.start = LeaderLine.pointAnchor(el_start, anchorSourceProps);
line.end = LeaderLine.pointAnchor(el_end, anchorTargetProps);

Thanks for your replies!

:smile:


UVA-EpiHiper submission for Phase 1

Description
UVA-EpiHiper submission for Disparities Round Phase 1: projections, abstract, and model metadata.

Hi @chenj-uva ,
Thank you for the submission, I just want to confirm I understand correctly the pairing information, so you submitted 300 samples, grouped by race_ethnicity, target and horizon, with each sample from the same location sharing the same run_grouping. Is that correct?
Please let me know if any questions or issues,
Thanks,
Lucie

Hi @LucieContamin
Yes correct. Thanks.
Jiangzhuo Chen


Force bazel to use d8 instead of dx for Android

bazel 5 by default uses dx which is deprecated since 2018 and no longer available in buildtools v31 and later.  This change upgrade bazel version to lates v5 release (5.4.1) and added command line flags to .bazelrc to force it to use d8 instead of dx. The hard coded buildtools version number is removed.

https://github.com/bazelbuild/rules_scala#bazel-compatible-versions

Tested on Ubuntu with sdk33 & ndk21d.

build perftest
test djinni-java-tests
All good.


Task registration error when specifying UserId of format user@domain

Describe the bug
The error occurs in the domain environment. When a task is logged on a remote computer using the Invoke-Command in powershell.
The task registers with an interactive token and specifies the user name in user@domain format.
If you register the task by specifying the username in domain\user format, the task registers successfully.
System.Security.SecurityException: A specified logon session does not exist. It may already have been terminated.
at System.Security.Principal.WindowsIdentity.KerbS4ULogon(String upn, SafeAccessTokenHandle& safeTokenHandle)
at System.Security.Principal.WindowsIdentity..ctor(String sUserPrincipalName, String type)
at System.Security.Principal.WindowsIdentity..ctor(String sUserPrincipalName)
at Microsoft.Win32.TaskScheduler.User..ctor(String userName)
at Microsoft.Win32.TaskScheduler.TaskFolder.RegisterTaskDefinition(String path, TaskDefinition definition, TaskCreation createType, String userId, String password, TaskLogonType logonType, String sddl)
To Reproduce
Run the command on the remote computer using Invoke-Command in powershell.
Invoke-Command -ComputerName TestServer -ScriptBlock {
    [Reflection.Assembly]::LoadFile("C:\bin\Microsoft.Win32.TaskScheduler.dll")

    $ts = [Microsoft.Win32.TaskScheduler.TaskService]::Instance

    $taskDefinition = $ts.NewTask()
    $taskDefinition.Actions.Add([Microsoft.Win32.TaskScheduler.ExecAction]::new("powershell"))
    $taskDefinition.Principal.RunLevel = [Microsoft.Win32.TaskScheduler.TaskRunLevel]::LUA
    $taskDefinition.Settings.Compatibility = [Microsoft.Win32.TaskScheduler.TaskCompatibility]::V2_2
    $taskDefinition.Settings.DisallowStartIfOnBatteries = $false
    $taskDefinition.Settings.ExecutionTimeLimit = [TimeSpan]::FromMinutes(5)

    $ts.RootFolder.RegisterTaskDefinition("Test",
        $taskDefinition,
        [Microsoft.Win32.TaskScheduler.TaskCreation]::CreateOrUpdate,
<EMAIL_ADDRESS>        [NullString]::Value,
        [Microsoft.Win32.TaskScheduler.TaskLogonType]::InteractiveToken)
}

Expected behavior
Successful task registration.
Screenshots

Environment (please complete the following information):

Host Windows version (the one running the code): Windows Server 2019
Library version: 2.9.2
Remote Windows version (if applicable): Windows Server 2019

Additional context
The error presumably occurs in the User class constructor on line 46 https://github.com/dahall/TaskScheduler/blob/270ba8f3553d003d0bf9776657d7ff5b461f0786/TaskService/User.cs#L46
It is most likely related to "double hop".

If the AdminUser user from domain1.local logs on locally on computer server1 in domain2.local and tries to register a task with an interactive token for the TestUser user, the error A specified logon session does not exist. It may already have been terminated.

[Reflection.Assembly]::LoadFile("C:\bin\Microsoft.Win32.TaskScheduler.dll")

$ts = [Microsoft.Win32.TaskScheduler.TaskService]::Instance

$taskDefinition = $ts.NewTask()
$taskDefinition.Actions.Add([Microsoft.Win32.TaskScheduler.ExecAction]::new("powershell"))
$taskDefinition.Principal.RunLevel = [Microsoft.Win32.TaskScheduler.TaskRunLevel]::LUA
$taskDefinition.Settings.Compatibility = [Microsoft.Win32.TaskScheduler.TaskCompatibility]::V2_2
$taskDefinition.Settings.DisallowStartIfOnBatteries = $false
$taskDefinition.Settings.ExecutionTimeLimit = [TimeSpan]::FromMinutes(5)

$ts.RootFolder.RegisterTaskDefinition("Test",
    $taskDefinition,
    [Microsoft.Win32.TaskScheduler.TaskCreation]::CreateOrUpdate,
<EMAIL_ADDRESS>    [NullString]::Value,
    [Microsoft.Win32.TaskScheduler.TaskLogonType]::InteractiveToken)

The same thing happens when executing code from a console application written in C#. Examples are given for powershell, as they are easier to reproduce.

@dahall ?

Do you have the same problem if you use the Register-ScheduledTask PowerShell cmdlet? I don't have a way to replicate and test this, so I first want to rule out that it isn't a problem with the underlying Windows Task Scheduler API. The error appears to be in Microsoft's WindowsIdentity class when calling a Kerberos authentication. The cmdlet from Microsoft may have approached this authentication differently, so please try it in your environment and let me know.


Fixed typo

Oh, I just realized you're french and that resource has two s's in it in French. So this isn't a typo in that case. I'll close this PR.


3.0.0.dev3 TypeError: Flux does not currently support ShellBolts with multiple streams.

sparse run
Cleaning from prior builds...
Creating topology Uber-JAR...
Uber-JAR created: /home/darkless/projects/sparse_testing/test/wordcount/_build/wordcount-0.0.1-SNAPSHOT-standalone.jar
Removing _resources temporary directory...done
Traceback (most recent call last):
  File "/home/darkless/.virtualenvs/sparse3/bin/sparse", line 11, in <module>
    sys.exit(main())
  File "/home/darkless/.virtualenvs/sparse3/local/lib/python2.7/site-packages/streamparse/cli/sparse.py", line 56, in main
    args.func(args)
  File "/home/darkless/.virtualenvs/sparse3/local/lib/python2.7/site-packages/streamparse/cli/run.py", line 85, in main
    debug=args.debug)
  File "/home/darkless/.virtualenvs/sparse3/local/lib/python2.7/site-packages/streamparse/cli/run.py", line 49, in run_local_topology
    topology_flux_dict = topology_class.to_flux_dict(name)
  File "/home/darkless/.virtualenvs/sparse3/local/lib/python2.7/site-packages/streamparse/dsl/topology.py", line 254, in to_flux_dict
    flux_dict['spouts'].append(cls._spec_to_flux_dict(spec))
  File "/home/darkless/.virtualenvs/sparse3/local/lib/python2.7/site-packages/streamparse/dsl/topology.py", line 205, in _spec_to_flux_dict
    .format(spec))
TypeError: Flux does not currently support ShellBolts with multiple streams. Given: ShellSpoutSpec(XWordSpout, par=1, name='word_spout', inputs={}, outputs={'second': StreamInfo(output_fields=['word'], direct=False), 'first': StreamInfo(output_fields=['word'], direct=False)}, component_object=ComponentObject(shell=ShellComponent(execution_command='streamparse_run', script='wordcount.XWordSpout'), java_object=None, serialized_java=None), common=ComponentCommon(inputs={}, json_conf='{}', streams={'second': StreamInfo(output_fields=['word'], direct=False), 'first': StreamInfo(output_fields=['word'], direct=False)}, parallelism_hint=1), config='{}')

I have modified wordcount topology for better reproducibility.
Here is respective topologies/wordcount.py:
"""
Word count topology
"""

from streamparse import Grouping, Topology, Stream

from bolts.wordcount import WordCountBolt
from spouts.words import WordSpout

class WordCount(Topology):
    class XWordSpout(WordSpout):
        outputs = [
            Stream(fields=['word'],
                   name='first',),
            Stream(fields=['word'],
                   name='second',),
        ]

    word_spout = XWordSpout.spec()
    first_count_bolt = WordCountBolt.spec(name=b"wordcount_first", inputs={word_spout['first']: Grouping.SHUFFLE},
                                    par=2)
    second_count_bolt = WordCountBolt.spec(name=b"wordcount_second", inputs={word_spout['second']: Grouping.SHUFFLE},
                                    par=2)


Refresh Labs (March)

Module: 00
Lab/Demo: 00
Fixes # .
Changes proposed in this pull request:

One additional comment. It would be good to include steps to  implement the logic to submit a simple prompt (consisting of system and user messages), display the response, and then submit a new prompt, showing that there's no memory of past conversation. Then update the code to include the previous messages and responses in the prompt and show that a "follow up" question can now use the context of the previous conversation.


When buying a drink, the history is blank for some seconds

Intentional this time is used to request the updated history again.

The History is cleared
The History is rebuild from the API


feat(askscript): function type constructor

fun(returnType, arg1Type, arg2Type, ..., argNType)
Last argument can be rest in this case last argument is true:
fun(string, array(number), true) - same as (s: string, ...numbers: number[]) in TypeScript
Examples
const myToString: fun(string, int) = fun(n:int): string {
  n:toString
}

[ ] update documentation

This PR changes function type used in .ask files to a new syntax.
I think the new syntax is confusing, e.g. fun(int, int, int) doesn't look like the a 2-argument function.
Let's chat about this change.

Also, this PR goes beyond the priorities from our Roadmap.
Let's focus on our current priorities before we discuss this syntax change.

This PR changes function type used in .ask files to a new syntax.
I think the new syntax is confusing, e.g. fun(int, int, int) doesn't look like the a 2-argument function.
Let's chat about this change.

@czerwinskilukasz1 I'm all for creating some syntax sugar to make it easier to use, see #49

@mhagmajer , is this PR ready for review?  I'd say no, because it links issue #49 , which is about a syntax sugar (int, int) -> int, but there is not such syntax introduced in your change. Instead you use the confusing fun(int, int, int).
I'd convert it to a draft. Thoughts?

@czerwinskilukasz1 this is not the recommended syntax for function type. This is an implementation detail - all types need constructors which are function that result in the given type. Function is a type of value so we need a function constructor for it. We can then have additional syntax sugar for it.
Because types are not checked by runtime yet, I created this PR only for the tests to showcase the basic syntax for creating function types.
Examples of function constructors:
const a: int({ positive: true }) = 5

const b: fun(int, int) = fun (a: int): int { 5 }

Because types are not checked by runtime yet, I created this PR only for the tests to showcase the basic syntax for creating function types.
Examples of function constructors:
const a: int({ positive: true }) = 5

const b: fun(int, int) = fun (a: int): int { 5 }

// possible syntax sugar:
const c: (int) -> int = fun (a: int): int { 5 }

The ticket description as well as your description in this PR and our discussion in this PR were  about a different feature than you mentioned today.

@czerwinskilukasz1 can you explain please? I meant this PR to introduce the generic function type constructor

@czerwinskilukasz1 can you explain please? I meant this PR to introduce the generic function type constructor

Of course. The linked issue is about (int, int) -> int syntax:

Last argument can be rest in this case last argument is true:
fun(string, array(number), true) - same as (s: string, ...numbers: number[]) in TypeScript

I find fun(string, array(number), true) syntax complex and confusing. How about we add ... to AskScript too?

@czerwinskilukasz1 all syntax sugars are useful to make writing code easier :)


CORE Beta: Error adding first contact

My first time using Serenity. Excellent project !!!
I tried the Serenity v2.9.21 ASP.NET CORE Beta on VS2017CE with all latest updates. It built fine and ran on my Win10 PC. I added an organization, then added another organization reporting to the first org. Then I added a contact, then received this error:
FILE: ContactRepository.cs
CODE LINE 16:  return new MySaveHandler().Process(uow, request, SaveRequestType.Create);
EXCEPTION MESSAGE:
System.TypeInitializationException occurred
HResult=0x80131534
Message=The type initializer for 'Serenity.ComponentModel.EmailEditorAttribute' threw an exception.
Source=
StackTrace:
at Serenity.ComponentModel.EmailEditorAttribute.Validate(IValidationContext context)
at Serenity.Services.SaveRequestHandler3.ValidateFieldValues()    at Serenity.Services.SaveRequestHandler3.ValidateRequest()
at Serenity.Services.SaveRequestHandler3.Process(IUnitOfWork unitOfWork, TSaveRequest request, SaveRequestType requestType)    at POC4.Organization.Repositories.ContactRepository.Create(IUnitOfWork uow, SaveRequest1 request) in C:\Users\Brian\documents\visual studio 2017\Projects\POC4\POC4\POC4.Web\Modules\Organization\Contact\ContactRepository.cs:line 16
at POC4.Organization.Endpoints.ContactController.Create(IUnitOfWork uow, SaveRequest`1 request) in C:\Users\Brian\documents\visual studio 2017\Projects\POC4\POC4\POC4.Web\Modules\Organization\Contact\ContactEndpoint.cs:line 18
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__27.MoveNext()
Inner Exception 1:
ArgumentOutOfRangeException: Specified argument was out of the range of valid values.

Another meeting module issue, ok lets keep them open.


helm init is failing with requested resource not found

I executed helm init and I got
Creating /Users/rcamara/.helm
Creating /Users/rcamara/.helm/repository
Creating /Users/rcamara/.helm/repository/cache
Creating /Users/rcamara/.helm/repository/local
Creating /Users/rcamara/.helm/plugins
Creating /Users/rcamara/.helm/starters
Creating /Users/rcamara/.helm/cache/archive
Creating /Users/rcamara/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://<IP_ADDRESS>:8879/charts
$HELM_HOME has been configured at /Users/rcamara/.helm.
Error: error installing: the server could not find the requested resource                                                                                exit:1

I ran this command kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://kubernetes.docker.internal:6443
  name: docker-desktop
- cluster:
    certificate-authority: /Users/rcamara/.minikube/ca.crt
    server: https://<IP_ADDRESS>:8443
  name: minikube
contexts:
- context:
    cluster: docker-desktop
    user: docker-desktop
  name: docker-desktop
- context:
    cluster: docker-desktop
    user: docker-desktop
  name: docker-for-desktop
- context:
    cluster: minikube
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: docker-desktop
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: minikube
  user:
    client-certificate: /Users/rcamara/.minikube/client.crt
    client-key: /Users/rcamara/.minikube/client.key

Helm version is
$ helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Error: could not find tiller

kubectl version
╰──➤  kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"clean", BuildDate:"2018-09-28T15:20:58Z", GoVersion:"go1.11", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

I'm using mac osx
Darwin M-C02SQ8WFGTF1 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64

Same error in Windows 10
C:\WINDOWS\system32>helm init
$HELM_HOME has been configured at C:\Users\gvc.helm.
Error: error installing: the server could not find the requested resource
C:\WINDOWS\system32>helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Error: could not find tiller
C:\WINDOWS\system32>kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

Hm, I'm hitting the same issue on macOS.
$ helm init

$HELM_HOME has been configured at /Users/tkg/.helm.
Error: error installing: the server could not find the requested resource

$ helm version

Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Error: could not find tiller

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T18:55:03Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

$ minikube version

minikube version: v1.4.0
commit: 7969c25a98a018b94ea87d949350f3271e9d64b6

I am facing the same issue.  I think, this issue is coming with K8s version 1.16.0. Till K8s version 1.15.3 it was ok.
If we take out the json file, helm init --output json , following parameter needs to be correct with respect to work with K8s version 1.16

"apiVersion": "extensions/v1beta1" => it should be "apiVersion: apps/v1"
"spec": {
"replicas": 1,
"selector": {
"matchLabels": {
"name": "tiller"
}
},

Hey there, this issue has already been reported in https://github.com/helm/helm/issues/6374. I'm going to close this issue as a duplicate so we can keep the conversation in one place. Please follow up in that ticket and use the workarounds provided in this comment: https://github.com/helm/helm/issues/6374#issuecomment-533185074
Thanks!

i also face this issue and i use the command in #6374  which is "helm init --service-account tiller --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | sed 's@  replicas: 1@  replicas: 1\n  selector: {"matchLabels": {"app": "helm", "name": "tiller"}}@' | kubectl apply -f -" , although helm problem will be get solved, but it also has some effects , for example minikube start problems.
finally, i find that  the client version and server version of kubectl must be the same.  so make the version match and this issue will be solved.

i also face this issue and i use the command in #6374 which is "helm init --service-account tiller --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | sed 's@ replicas: 1@ replicas: 1\n selector: {"matchLabels": {"app": "helm", "name": "tiller"}}@' | kubectl apply -f -" , although helm problem will be get solved, but it also has some effects , for example minikube start problems.
finally, i find that the client version and server version of kubectl must be the same. so make the version match and this issue will be solved.

It worked for me with the following:

kubectl apply -f tiller.yaml

helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | kubectl apply -f -

Tiller.yaml:

kind: ServiceAccount
apiVersion: v1
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

faced same issue while connecting to AKS cluster, while I had access of cluster through Kubectl command.
Second command provided by @sergiomartins8  solved issue.


Remove package-lock.json and update contributing.md to reflect change

As referenced in #74
I also snuck an unrelated section about opening the dev tools in there. I was already in the file and didn't think it was quite worth its own PR.

I'm full of typos today...

And a missing new line... I'm so used to prettier doing its thing by now.

Thanks for the review. I really appreciate it.

Sorry I didn't catch this before it was merged, but your description of the DevTools isn't quite right. Ctrl+h toggles their visibility (shows and hides) while Ctrl+q changes which edge of the screen they're docked to.

Good catch, @superhawk610! @karlsander would you mind creating a new PR that updates this?


Adding VIES VAT api

src/ViesClient.php (blank_line_after_namespace) looks like there was a validation error. PR looks good to me otherwise


Change syncBranch to master branch

Changed sync branch to master branch since CSR default isn't main.

While the CSR UI defaults to master, we are trying to promote usage of main instead to follow new industry practices. @bharathkkb @kaariger Thoughts?

From a CSR perspective, it just means when the page loads there is a warning and asks user to switch branches. I am also planning to switch to main in example foundation (https://github.com/terraform-google-modules/terraform-example-foundation/issues/486) so I think we should continue using main here and maybe provide a setter in the future if there is demand.

While the CSR UI defaults to master, we are trying to promote usage of main instead to follow new industry practices. @bharathkkb @kaariger Thoughts?

I have internally been using main as well. I do think it's good to adopt the new recommendation and avoid master.

@tyorikan Thank you for the PR, but at this time I think we're going to continue preferring main as the default. You can of course override it in any instances you check out though.


Add TShirt size field to all tasks

TO better orchestrate estimates all tasks need a size estimate
Sizes Here
https://www.c-sharpcorner.com/article/agile-story-point-estimation-techniques-t-shirt-sizing/#:~:text=What is T-shirt sizing,%2C M%2C L%2C XL.
Action Items
-Add new single-select field to Github

[ ] S - Small
[ ] M - Medium
[ ] L - Large
[ ] XL - Extra Large
-Pick nice colors for each of the above

Comment 1

Comment 2

Comment 3
Weeeee!!!

This is the end


Have optimum supported BLIP-2 model converted to onnx?

Hi, have optimum supported BLIP-2 model converted to onnx?

Hi @joewale , it is currently not supported. I added it to the list of to-be-done ONNX support: https://github.com/huggingface/optimum/issues/555. Feel free to submit a PR or rerequest the support in the linked thread!


Broken link to list of node aliases

Plugins - Visiting page links to https://github.com/babel/babel/blob/master/src/babel/types/alias-keys.json in "Visitor aliases" section which was either moved or doesn't exist anymore.

Did a PR for this in #387

Yup, merged


Fix cc extractor (which references mtv extractor)

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

[X] At least skimmed through contributing guidelines including yt-dlp coding conventions
[X] Searched the bugtracker for similar pull requests
[X] Checked the code with flake8

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[X] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[X] Bug fix
[ ] Improvement
[ ] New extractor
[ ] New feature

Description of your pull request and other information
Recent changes to cc.com causes failures for downloading videos. I added some additional checking and regex checks for the extractor to address these changes.

https://github.com/ytdl-org/youtube-dl/pull/30149

@Sipherdrakon can u have a look at how this and the upstream PR interact with your partial mtv fix

So I finally had time to check the changes made here and within youtube-dl. I checked every site attached to the mtvextractor and found that some parts of some of the sites were unsupported and that the PR upstream works far better than this PR, but it isn't perfect.
I'm not sure how to make changes to this PR but I can make a new one to fix the upstream PR to work with all supported URLs that were tested.

@Sipherdrakon You can open a new PR


Develop

Issue #, if available: "Implement SNS filter for Lambda trigger"
Description of changes:

Replace the filtering Glue jobs in Lambda Function "StateTopicSubscription". Delete related SNS Topic. Instead use EventBridge rule to filter the Glue jobs.

Replace default DBName "meter-data" by "meter_data"

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

thanks, tested and merged.


[cxx-interop] Add diagnostic notes when retain/release are not import…

If the retain or release function of a C++ foreign reference type (or SWIFT_SHARED_REFERENCE type) was not imported into swift because of any reasons, then we were just emitting this error:
cannot find retain/release function '<retain/release function name>' for reference type ''.
This is confusing to a C++ developer because the retain/release function clearly exists on C++ side. It is just not imported into swift because it violated some contract.
In this patch I am adding additional notes in such cases when retain/release functions are actually present on the C++ side, but are not imported into swift for whatever reason.
rdar://136834573

@swift-ci please smoke test

@swift-ci please smoke test


microk8s in LXD

Required information

Distribution: Ubuntu
Distribution version: Xenial
The output of "lxc info" or if that fails:

config:
  core.https_address: '[::]:4443'
  core.trust_password: true
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- candid_authentication
- backup_compression
- candid_config
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- candid_config_key
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses:
  - 176.xxx.xxx.xxx:4443
  - <IP_ADDRESS>:4443
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----

    -----END CERTIFICATE-----
  certificate_fingerprint:   driver: lxc
  driver_version: 3.1.0
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 4.4.0-141-generic
  server: lxd
  server_pid: 32108
  server_version: "3.11"
  storage: btrfs
  storage_version: "4.4"
  server_clustered: false
  server_name: ubuntu-xenial
  project: default

Issue description
I´m trying to spin up mikrok8s in a bionic lxd container. The snap installation succeeds but kubelet service is not starting:
microk8s.inspect
Inspecting services
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-proxy is running
 FAIL:  Service snap.microk8s.daemon-kubelet is not running
For more details look at: sudo journalctl -u snap.microk8s.daemon-kubelet
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Service snap.microk8s.daemon-etcd is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system info
  Copy network configuration to the final report tarball
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Inspect kubernetes cluster

microk8s.kubectl get all --all-namespaces
NAMESPACE     NAME                                                  READY   STATUS    RESTARTS   AGE
kube-system   pod/heapster-v1.5.2-5c5498f57c-zgcxr                  0/4     Pending   0          40m
kube-system   pod/kube-dns-6bfbdd666c-sgvl6                         0/3     Pending   0          40m
kube-system   pod/kubernetes-dashboard-6fd7f9c494-l5vsw             0/1     Pending   0          40m
kube-system   pod/monitoring-influxdb-grafana-v4-78777c64c8-wwtgp   0/2     Pending   0          40m

NAMESPACE     NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
default       service/kubernetes             ClusterIP   <IP_ADDRESS>     <none>        443/TCP             54m
kube-system   service/heapster               ClusterIP   <IP_ADDRESS>    <none>        80/TCP              40m
kube-system   service/kube-dns               ClusterIP   <IP_ADDRESS>    <none>        53/UDP,53/TCP       40m
kube-system   service/kubernetes-dashboard   ClusterIP   <IP_ADDRESS>   <none>        443/TCP             40m
kube-system   service/monitoring-grafana     ClusterIP   <IP_ADDRESS>   <none>        80/TCP              40m
kube-system   service/monitoring-influxdb    ClusterIP   <IP_ADDRESS>   <none>        8083/TCP,8086/TCP   40m

NAMESPACE     NAME                                             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/heapster-v1.5.2                  0/1     1            0           40m
kube-system   deployment.apps/kube-dns                         0/1     1            0           40m
kube-system   deployment.apps/kubernetes-dashboard             0/1     1            0           40m
kube-system   deployment.apps/monitoring-influxdb-grafana-v4   0/1     1            0           40m

NAMESPACE     NAME                                                        DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/heapster-v1.5.2-5c5498f57c                  1         1         0       40m
kube-system   replicaset.apps/kube-dns-6bfbdd666c                         1         1         0       40m
kube-system   replicaset.apps/kubernetes-dashboard-6fd7f9c494             1         1         0       40m
kube-system   replicaset.apps/monitoring-influxdb-grafana-v4-78777c64c8   1         1         0       40m

microk8s.kubectl cluster-info
Kubernetes master is running at https://<IP_ADDRESS>:16443
Heapster is running at https://<IP_ADDRESS>:16443/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://<IP_ADDRESS>:16443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Grafana is running at https://<IP_ADDRESS>:16443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
InfluxDB is running at https://<IP_ADDRESS>:16443/api/v1/namespaces/kube-system/services/monitoring-influxdb:http/proxy

root@microk8s:~# microk8s.kubectl get nodes
NAME       STATUS     ROLES    AGE   VERSION
microk8s   NotReady   <none>   56m   v1.14.0

The mikrok8s logs are quite chatty but here are some lines that might cause the issue:
364 fs.go:556] stat failed on /dev/dm-0 with error: no such file or directory
Apr  8 11:31:31 microk8s microk8s.daemon-kubelet[1364]: F0408 11:31:31.078576    1364 kubelet.go:1359] Failed to start ContainerManager failed to get rootfs info: failed to get device for dir "/var/snap/microk8s/common/var/lib/kubelet": could not find device with major: 0, minor: 272 in cached partitions map
Apr  8 11:31:31 microk8s systemd[1]: snap.microk8s.daemon-kubelet.service: Main process exited, code=exited, status=255/n/a
Apr  8 11:31:31 microk8s systemd[1]: snap.microk8s.daemon-kubelet.service: Failed with result 'exit-code'.
Apr  8 11:31:31 microk8s systemd[1]: snap.microk8s.daemon-kubelet.service: Service hold-off time over, scheduling restart.
Apr  8 11:31:31 microk8s systemd[1]: snap.microk8s.daemon-kubelet.service: Scheduled restart job, restart counter is at 6.
Apr  8 11:31:31 microk8s systemd[1]: Stopped Service for snap application microk8s.daemon-kubelet.
Apr  8 11:31:31 microk8s systemd[1]: snap.microk8s.daemon-kubelet.service: Start request repeated too quickly.
Apr  8 11:31:31 microk8s systemd[1]: snap.microk8s.daemon-kubelet.service: Failed with result 'exit-code'.
Apr  8 11:31:31 microk8s systemd[1]: Failed to start Service for snap application microk8s.daemon-kubelet.

The container config looks like this:
architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 18.04 LTS amd64 (release) (20190402)
  image.label: release
  image.os: ubuntu
  image.release: bionic
  image.serial: "20190402"
  image.version: "18.04"
  linux.kernel_modules: ip_tables,ip6_tables,netlink_diag,nf_nat,overlay
  raw.lxc: |
    lxc.apparmor.profile = unconfined
    lxc.cgroup.devices.allow = a
    lxc.mount.auto=proc:rw sys:rw
    lxc.cap.drop =
  security.nesting: "true"
  security.privileged: "true"
  volatile.base_image: 663f6663aed66a22dd708c4b07514748221522b810008c55002fcc1dd81af377
  volatile.eth0.hwaddr: 00:16:3e:d2:3c:28
  volatile.idmap.base: "0"
  volatile.idmap.next: '[]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""

root@microk8s:~# snap list
Name      Version  Rev   Tracking  Publisher   Notes
core      16-2.38  6673  stable    canonical✓  core
microk8s  v1.14.0  492   stable    canonical✓  classic

I also tried the edge branch but with the same result.
I´m not sure if this is a issue related to LXD or microk8s.
Thanks.

Yep, that's a known issue with microk8s and something the Kubernetes team at Canonical is working on.
The intent is that you should only need to set:

security.privileged=true
security.nesting=true
linux.kernel_modules=ip_tables,ip6_tables,netlink_diag,nf_nat,overlay

And then everything will work fine. Right now kubelet is trying to be smart about storage and failing.
@ktsakalozos may be able to provide more details.
There's nothing for us to do about this on the LXD side, so closing the issue here.

Hi @huegelc,
As @stgraber mentioned the intent is to have MicroK8s working on LXC with a minimal profile. We have a PR upstream that works towards this direction but it has not landed yet. For now, to have MicroK8s on LXC we use the two profiles you can find here: https://github.com/ubuntu/microk8s/tree/master/tests/lxc

Thank you, much appreciated.

Hi @huegelc,
As @stgraber mentioned the intent is to have MicroK8s working on LXC with a minimal profile. We have a PR upstream that works towards this direction but it has not landed yet. For now, to have MicroK8s on LXC we use the two profiles you can find here: https://github.com/ubuntu/microk8s/tree/master/tests/lxc

@ktsakalozos What is the link to that upstream issue? I cannot tell if this is supposed to work yet or not. It would be useful to track that issue.

Sorry for the late reply @drewboswell. I totally missed this.
The PR I was referring to is [1]. In the meantime we have revised the MicroK8s docs with instructions on how to install it on lxc [2].
[1] https://github.com/google/cadvisor/pull/2189
[2] https://microk8s.io/docs/lxd

Thanks @ktsakalozos super helpful!


Add release notes for 0.14.1

https://github.com/submariner-io/releases/issues/559 + https://github.com/submariner-io/subctl/pull/435

@maayanf24 can you handle this please?

@maayanf24 can you handle this please?

Sure

PR: https://github.com/submariner-io/submariner-website/pull/863


Add in flag for acceptance testing

This addresses the first part of #1421
I am going to see if pre-commit tests pass first and then look into updating other test instances.

build

build

@jlowe the pre-commit build is failing on the native UDFs not being configured to run.  I think that is expected because we didn't want to require hive to build/run for these tests?  If not I can try and fix it. If so do we want to adjust this so we say what type of tests are being run instead (AKA precommit, nightly, developer)?

@jlowe the pre-commit build is failing on the native UDFs not being configured to run.  I think that is expected because we didn't want to require hive to build/run for these tests?  If not I can try and fix it. If so do we want to adjust this so we say what type of tests are being run instead (AKA precommit, nightly, developer)?

I think that is expected because we didn't want to require hive to build/run for these tests?

No, pretty sure Hive is being leveraged by other integration tests during precommit runs.
There are two flavors of Hive UDF example tests: Java and native.  The latter requires building in an environment where libcudf was also built which is time consuming and IMO unnecessary for a precommit build.  Note that without the native example test we'll still be exercising the plugin code around Hive UDFs via the other tests that don't require a native code environment to build the example code (e.g.: URLDecode and URLEncode tests which are not tied to the rapids_udf_example_native tests option).

If so do we want to adjust this so we say what type of tests are being run instead (AKA precommit, nightly, developer)?

Might be useful for tests that are too involved or time consuming to warrant running during precommit (like the Hive UDF native code example test).  I don't think it's required for this PR, we could track tackling the native UDF test in a followup.

I think that is expected because we didn't want to require hive to build/run for these tests?

No, pretty sure Hive is being leveraged by other integration tests during precommit runs.
There are two flavors of Hive UDF example tests: Java and native.  The latter requires building in an environment where libcudf was also built which is time consuming and IMO unnecessary for a precommit build.  Note that without the native example test we'll still be exercising the plugin code around Hive UDFs via the other tests that don't require a native code environment to build the example code (e.g.: URLDecode and URLEncode tests which are not tied to the rapids_udf_example_native tests option).

If so do we want to adjust this so we say what type of tests are being run instead (AKA precommit, nightly, developer)?

Might be useful for tests that are too involved or time consuming to warrant running during precommit (like the Hive UDF native code example test).  I don't think it's required for this PR, we could track tackling the native UDF test in a followup.

build

build

@jlowe I have updated the tests to know what kind of tests are being run.

@jlowe I have updated the tests to know what kind of tests are being run.

build

build

build

build


Copr: Use github status, instead of comments for reporting

There is an example in checks of this PR, there seems to be some problem with man page generating :)

The status is very nice, but sometimes the comments can be useful as well. (Just look to pull request on flathub, where you get instruction, how you can install the build version.

Merging and will create a follow-up with the instructions.

Congratulations! The build has finished successfully. :champagne:
You can install the built RPMs by following these steps:

sudo yum install -y dnf-plugins-core on RHEL 8
sudo dnf install -y dnf-plugins-core on Fedora
dnf copr enable dhodovsk/packit
And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.


Enhancement: Allow configuration using system properties

As I am using the plugin in a GitLab CI/CD component it is sometimes useful to execute the plugin without any changes to the pom.xml.
The possibility to configure the plugin using system properties would be great to have.

Fixed via #6. Thanks a lot for your help. I'll try to find some time over the weekend to publish a new release!

I just released version 1.1.0 to Maven Central. It may take a few hours for the release to show up there.
Thanks again for your help!

Thanks a lot for your effort. Glad I could Help.


Close Container From DataStore

Feature
We currently allow datastores to call raiseContainerWarning, but all this does is log and raise an event. In the event of a fatal error there isn't a way for a datastore to close a container.
Right now the only option is for a host to close a container on the container warning event, but this depends on the host doing to right thing depending on the error, and raiseContainerWarning is not called consistently making the right things hard to decide

@vladsud and @curtisman curious what your thoughts are here.
an option could be to expose a container runtime scope for this. what i like about that is it up to the container Runtime if components have the power to close the container. it also opens the door for other actions, like making the container readonly, rather than closing

There is ContainerRuntime.closeFn that is exposed through IContainerRuntime in some form. I have not looked at it for a while, this can be used to close container, may need some plumbing to make it accessible easily (if it's not on IContainerRuntimeBase).
I personally do not like global things like that (i.e. exposing on IContainerRuntimeBase, especially general purpose scope object that can do a lot of things), I think it's much better (even if it requires much more typing) for specific container runtime instance to make decision to expose (or not) some custom interface to object beneath it (data stores). And either have a way to provide it on both create & load, or provide it on create but make it serializable, such that data store could store it in its data model (DDSs) and rehydrate it later on load. That way it's a contract that someone made a decision to expose to specific user, same as in "normal programming" flow (i.e. essentially - avoid globals, pass things as explicit arguments to constructor).

I only see closeFn expose off the container context interface, not on any of the runtime interfaces:
https://github.com/microsoft/FluidFramework/blob/9092364501773d9fc622abc864a8307dff87f669/common/lib/container-definitions/src/runtime.ts#L110
container runtime does implement it: https://github.com/microsoft/FluidFramework/blob/1f3c3f8500d17f29c05f05a66278183696c60782/packages/runtime/container-runtime/src/containerRuntime.ts#L673
so it could be accessed with a case.


Support 2D array serialization

Support 2D array serialization
Add unit tests to demonstrate 1D and 2D array serialization
Fix #80

What about serializing this as a list of lists?
Can we scale this up to n-dimensional arrays?

I thought about a list of lists, but decided against it because it seemed more appropriate for [][] arrays rather than [,] arrays, and because it adds to the serialized size. Not a big deal I know!
It certainly could be scaled up to n-dimensional arrays - I just wrote this because my object graph had 2D arrays so I needed it, I haven't encountered any 3+ dimensional arrays. It's up to you if you merge or not of course - I probably won't generalise this until I need it though, sorry :/ (total lack of free time!)

Thinking about it more, I think I prefer serializing as a list of lists because it lets you swap swap [][] and [,] interchangeably without creating a versioned model.
I'll be happy to accept the PR if you change the serialization model to the list of lists. I'm fine with only 2D array support for now, but if you find the time to generalize it that'd be great :).

Oh, I remember the actual reason now - it's because [,] is always rectangular, whereas [][] is an array of arrays where each inner array could be a different length.

I think it's still worth adding the additional logic in that scenario, since having compatible serialization data is a very compelling argument. I think that if dimensions don't match up in the JSON file, then set the missing elements to the default and emit a warning.

Is this in a released version anywhere?

Nope - I never found the time (or need) to make the additional changes to get this PR accepted... I'd just fork this repo, add mine (and others) as remotes, and create your own branch merging whichever feature branches you fancy, that's what I've done :)


Using std::expected if compiled with C++23

I am suggesting that this library be made up to date with newer standards. Hence
For people compiling with c++20, we can either :

use rfl::Result
or
use std::expected by including the header
And for people compiling on C++23 or higher :
use std::expected<..., rfl::Error> for all.

This is just a discussion, but adding this will mean we will need to use macros but its use can be localized to only one file and exporting something like
using rfl::Result = ...

@jowillianto , yeah, I was thinking about this as well. In fact, I designed rfl::Result with std::expected in mind.
We could simply use __cpp_lib_expected:
#ifdef __cpp_lib_expected

template<class T>
using rfl::Result = std::expected<T, rfl::Error>;

#else

...

#endif

Should I look to implement it ?

@jowillianto why do you think that? I don't think you have to wrap it in an optional.
Sure, go ahead, implement it.

@jowillianto, I see what you mean...because calling error() on std::expected when it actually contains a  value is undefined behaviour whereas we just return std::nullopt, so it is very defined.
I think you could just adapt that to the standard.

Ah I see. So that is the intention, changing semantics could break backwards compatibility, how should we act on this issue ?
I.e.
using rfl::Error vs std::optionalrfl::Error is two different semantics

Ok

If you have any questions, let me know

@jowillianto have you already worked on this, or are you planning to work on it soon?
If not, i would work on it on the weekend.

I am working on it !

Hello !
I have done the conversion and run the tests for C++20. Now onto C++ 23. Please review Result.hpp to check if the API design is as what we all wanted.

When I am working on C++23 version, I have to make the following adjustments :
#ifdef __cpp_lib_expected

template<class T>
struct Result : public std::expected<T, rfl::Error> {
  Result(rfl::Error err) : std::expected<T, rfl::Error>{std::unexpected{std::move(err) }} {}
  template <typename ...Args> requires (std::is_constructible_v<std::expected<T, rfl::Error>, Args...>)
  Result(Args&& ...args> : std::expected<T, rfl::Error>{std::forward<Args>(args)...}{}
};

#else

...

#endif

This ensures that Result is assignable to std::expected and circumvent issues related to std::expected requiring construction with std::unexpected for unexpected cases.

@jowillianto , sounds great!
Could you open a PR? It would be easiest to discuss your changes there.

I am not done yet. I need to further develop this ! Should I open a PR just for discussion ?

@jowillianto, sorry, I misread what you wrote. I thought you were done and wanted me to review stuff.
Open a PR whenever you want me to review your code. You can open it as a draft.
But yes, I think we need some tests running on C++23. I would also like to see some benchmarks.

@liuzicheng1987 since I am new as a contributor, I want to ask about what kind of intellisense are you using ? My clangd keeps failing and marking things up as an error even though it compiles just fine or is there a configuration that I am not aware of ?

can wer

I also use clangd...you sometimes get false positives, it's normal.

I also use clangd...you sometimes get false positives, it's normal.

Thanks !


Some recipes have a separate serve with section for ingredients

Might make sense to separate these so the information can be displayed more nicely.
Additionally, it might be the case that there are multiple sets of directions for different parts of a meal.
Example: https://www.simplyrecipes.com/recipes/slow_cooker_taco_soup/

Another example of this: https://www.seriouseats.com/recipes/2017/02/detroit-style-pizza-recipe.html


Copyright year in created package still says 2017

As per the title.
e.g. :copyright: Copyright 2017 by Dominic Davis-Foster even though I've only just run cookiecutter.
I assume the year is hardcoded? Could it be reconfigured to get the current year and use that instead?

Further to that, the LICENSE file has the year 2018 at the top. Again, I assume that's hardcoded currently?


Non-zero CO2 surface flux for an ocean in equilibrium (with non-zero average wind speed)

Thank you everyone for the super nice package (especially @jagoosw)! I know everything is still in development with lots of changes happening.
I've been playing around with LOBSTER (well, really just the carbonate chemistry component) and the air-sea gas exchange model. I'm just initializing an ocean at equilibrium with constant T, S, DIC, and Alk. I set the atmospheric pCO2 equal to the ocean's pCO2. I expect a surface CO2 flux of zero. I see this but only when average_wind_speed = 0.
When average_wind_speed = 10.0 (the default) I believe the flux should still be zero since pCO2 is equal across the air-ocean interface but it's slightly negative after one time step. It's zero at time step 0.
Is this behavior expected or am I misunderstanding the gas exchange model?
Thank you!

Here's a minimal working example to showcase what I'm seeing:
using Oceananigans
using OceanBioME

using OceanBioME.Boundaries: OCMIP_default

const pCO₂_OCMIP_default = OCMIP_default

grid = RectilinearGrid(
    CPU(),
    topology = (Periodic, Flat, Bounded),
    size = (256, 128),
    x = (0, 200),
    z = (-100, 0),
    halo = (4, 4)
)

ambient_Alk = 2300.0
ambient_DIC = 2100.0
SST = 20.0
SSS = 35.0

# Let's set the atmospheric pCO₂ to equal the oceanic pCO₂.
pCO₂_atmosphere = pCO₂_OCMIP_default(ambient_DIC, ambient_Alk, SST, SSS)

CO₂_flux = GasExchange(;
    gas = :CO₂,
    air_concentration = pCO₂_atmosphere,
    average_wind_speed = 10.0
)

boundary_conditions = (DIC = FieldBoundaryConditions(top = CO₂_flux), )

model = NonhydrostaticModel(;
    grid,
    buoyancy = SeawaterBuoyancy(equation_of_state=LinearEquationOfState()),
    biogeochemistry = LOBSTER(; grid, carbonates=true),
    tracers = (:T, :S, :NO₃, :NH₄, :P, :Z, :sPOM, :bPOM, :DOM, :DIC, :Alk),
    boundary_conditions = boundary_conditions
)

set!(model, T=SST, S=SSS, Alk=ambient_Alk, DIC=ambient_DIC)

function surface_CO2_flux(model)
    grid = model.grid
    T_surface = interior(model.tracers.T, :, 1, grid.Nz)
    S_surface = interior(model.tracers.S, :, 1, grid.Nz)
    DIC_surface = interior(model.tracers.DIC, :, 1, grid.Nz)
    Alk_surface = interior(model.tracers.Alk, :, 1, grid.Nz)
    flux_func = model.tracers.DIC.boundary_conditions.top.condition.func
    return flux_func.(0, 0, 0, DIC_surface, Alk_surface, T_surface, S_surface)
end

Time step 0:
julia> surface_CO2_flux(model)
256-element Vector{Float64}:
 0.0
 0.0
 0.0
 0.0
 0.0
 ⋮
 0.0
 0.0
 0.0
 0.0
 0.0

Time step 1:
julia> time_step!(model, 1)

julia> surface_CO2_flux(model)
256-element Vector{Float64}:
 -8.941545089802037e-8
 -8.941545089802037e-8
 -8.941545089802037e-8
 -8.941545089802037e-8
 -8.941545089802037e-8
  ⋮
 -8.941545089802037e-8
 -8.941545089802037e-8
 -8.941545089802037e-8
 -8.941545089802037e-8
 -8.941545089802037e-8

I'm using Julia 1.10.4 with
  [a49af516] OceanBioME v0.10.5
  [9e8cae18] Oceananigans v0.91.4

Hi @ali-ramadhan, thank you for raising this issue!
I've had a look and am pretty sure this is because the grid is flat, so when Oceananigans calls getbc the arguments are x, t, DIC, Alk, T, S, but when I wrote this it would have passed x, 0, t, DIC, Alk, T, S, so its not computing the pCO2 in the water correctly first. I.e. its using the second method not the first one here:
https://github.com/OceanBioME/OceanBioME.jl/blob/4bcaf29797abcb7c7293a299c2e1c549e3d3d7eb/src/Boundaries/gasexchange.jl#L223-L232
I tried running your code with y periodic and it appears to be correct.
Currently, I'm doing a lot of work on the gas exchange model because I've discovered a few issues with how I originally implemented it, and following that PR (#186) it will be more sophisticated and better tested. This situation would also cause an issue with the refactored code so I'll work out how to fix it there.
For now, the best thing is probably to only use 3D grids I think.

Thanks for having a look @jagoosw! Tricky dispatch haha. It does work with a 3D grid and I was able to hack in a boundary condition wrapper for 2D grids.
gas_exchange = GasExchange(;
    gas = :CO₂,
    air_concentration = pCO₂_atmosphere
)

@inline CO₂_flux(x, t, DIC, Alk, T, S, ge) = ge.condition.func(x, 0, t, DIC, Alk, T, S)

DIC_top_bc = FluxBoundaryCondition(CO₂_flux, field_dependencies = (:DIC, :Alk, :T, :S), parameters = gas_exchange)

Out of curiousity, do you know why it works correctly when average_wind_speed = 0.0?

Just to chime in --- I recommend using the discrete form in packages that build out functionality for Oceananigans. That should avoid the issues that occur when trying to write generic code for Flat or 3D grids.

Out of curiousity, do you know why it works correctly when average_wind_speed = 0.0?

The flux is like $k(u_{wind})(pCO_{2, \text{air}} - pCO_{2, \text{water}})$ and the common form of $k(u_{wind})$ is $k_0u_{wind}^2(660/Sc)^{1/2}$ so it also goes to zero when the wind speed is zero. The new implementation allows the actual wind speed to be put in, as well as having different transfer velocities since a few have a constant factor was well.


WARNING: [youtube] Unable to fetch PO Token for mweb client: Missing required Visitor Data. You may need to pass Visitor Data with --extractor-args "youtube:visitor_data=XXX"

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
Hello guys. Although I use a proxy, my code constantly gives an error and I can't overcome this problem.
What this is meaning WARNING: [youtube] Unable to fetch PO Token for mweb client: Missing required Visitor Data. You may need to pass Visitor Data with --extractor-args "youtube:visitor_data=XXX" ?
youtube  | 1 - 2024-10-16 09:15:49,438 - INFO - Total comments: 31
youtube  | [youtube] GXm6AsKu1wM: Downloading ios player API JSON
youtube  | [youtube]     Downloading comment API JSON reply thread 1 (266/~334)
youtube  | [youtube]     Downloading comment API JSON reply thread 1 (1004/~1848)
youtube  | [youtube] GXm6AsKu1wM: Downloading mweb player API JSON
youtube  | [youtube]     Downloading comment API JSON reply thread 2 (285/~334)
youtube  | [youtube] Downloading comment API JSON page 46 (1009/~1848)
youtube  | [youtube] Downloading comment API JSON page 15 (286/~334)
youtube  | ERROR: [youtube] GXm6AsKu1wM: Sign in to confirm you’re not a bot. This helps protect our community. Learn more
youtube  | 1 - 2024-10-16 09:15:51,105 - ERROR - Failed to fetch comments after 3 attempts for Video URL: https://www.youtube.com/watch?v=GXm6AsKu1wM. Error: ERROR: [youtube] GXm6AsKu1wM: Sign in to confirm you’re not a bot. This helps protect our community. Learn more
Provide verbose output that clearly demonstrates the problem

[ ] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
No response

do not use a datacentre IP to access youtube. either rotate your ip(use your own residential one) or sign in as it prompted(your account may get banned after lots of downloads)

Please review https://github.com/yt-dlp/yt-dlp/wiki/Extractors#po-token-guide
Also note that PO tokens are not a solution to an IP block


Make sure that CUDA instances are destroyed prior to destroying streams

Fix #1049. Should fix nightly failures.

HIP failed, but this PR does not include #1048.

This indeed fixed CUDA nightlies.


Error: It's currently unsupported to use "export *" in a client boundary. Please use named exports instead.

I have a React hook where there is also query getter defined for an axios call. E.g.
import { useQuery } from '@tanstack/react-query';
import { useCookies } from 'next-client-cookies';

export async function fetchTasks(cookies): Promise<object[]> {
  try {
    const tasks =
      (await axios.get({ headers: { cookies }, url: '/api/tasks'))?.data || [];
    return tasks;
  } catch (err) {
    console.error({ error: err }, 'Failed to fetch onboarding tasks');
    return [];
  }
}

export function useTasks() {
  const cookies = useCookies();
  return useQuery(() => fetchTasks(cookies));
}

The import { useCookies } from 'next-client-cookies'; import statement causes application to crush with:
Failed to compile
./node_modules/next-client-cookies/dist/index.mjs
Error: It's currently unsupported to use "export *" in a client boundary. Please use named exports instead.
This error occurred during the build process and can only be dismissed by fixing the error.

I.e. this file is used in two places:
import { useTasks } from './useTasks';

export default function SomeClientComponent() {
    const { data: tasks = [] } = useTasks();
    return <>{tasks}</>
}

and also from a server page like:
import {
  HydrationBoundary,
  QueryClient,
  dehydrate,
} from '@tanstack/react-query';
import { getCookies } from 'next-client-cookies/server';

import { fetchTasks } from './useTasks';

export default async function SomePage() {
  const cookies = getCookies();
  const queryClient = new QueryClient();

  await queryClient.prefetchQuery({
    queryKey: tasksQueryKey(),
    queryFn: () => fetchTasks({ cookies }),
  });

  return (
    // Neat! Serialization is now as easy as passing props.
    // HydrationBoundary is a Client Component, so hydration will happen there.
    <HydrationBoundary state={dehydrate(queryClient)}>
      <SomeContent />
    </HydrationBoundary>
  );
}

Ah, I guess I could separate the file in the interim 🤷

Nope, that did NOT help. 🤔

Can you share a repo with reproduction of this issue?
We have an example of a working project: https://github.com/moshest/next-client-cookies/tree/main/demo
It's probably related to something within your configuration and not this package.

Just had the same error. You need to add "use client" at the top of the your client source.


setFieldsRows does not escape single quote properly

hi. i'm using squel version 4.2.3 and setFieldsRows does not escape single quote properly :
$ node
> require('squel').insert().into('demos').setFieldsRows([{column: "just a single quote' ouch"}]).toString()
'INSERT INTO demos (column) VALUES (\'just a single quote\' ouch\')'

on pg result should be something like :
INSERT INTO demos (column) VALUES ('just a single quote'' ouch')
-- OR
INSERT INTO demos (column) VALUES (E'just a single quote\' ouch')

Thanks for that report. I think the string formatting needs an overhaul generally.

Serious safety bug! Please fix!

From what I can see this usage should be fine if you use toParam() instead of toString(), as you end up with:
require('squel').insert().into('demos').setFieldsRows([{column: "just a single quote' ouch"}]).toParam();
{ text: 'INSERT INTO demos (column) VALUES (?)',
  values: [ 'just a single quote\' ouch' ] }

And the DB layer should safely handle the quote in that situation.
Is usage of toString() generally considered unsafe with the lib?
The docs do say:
"In general it is better to use toParam() rather than toString() when constructing the final query as it gives you the opportunity to use your database engine methods to safely escape and interpolate your parameters."
Although given this behaviour, perhaps a more prominent warning should be given, as most of the examples use toString for convenience of producing a final SQL string which makes the tranform easier to see in the examples.

Looks like .toString() should escape quotes (maybe with some additional flag), if it's not, it requires additional wrapping just to use library, that weird. Why should I wrap query builder to build safe queries? It's basic functionality, isn't it?

I'll add this in for the next milestone. The plan is to provide the ability to override the string value formatter with a custom function, allowing you to escape things any way you want.


Performance Optimizations for coupling metrics (--parseDependencies)

some notes about that can be found in main readme file.

Without benchmarking, I also expect the data structures being used as being highly problematic.
Right now, multiple data structures are copied each time an entry is added.
https://github.com/MaibornWolff/metric-gardener/blob/f5e498fec945f58d07565c813630bb57ca366a62/src/parser/metrics/coupling/Coupling.ts#L41-L44
Should be e.g.:
for (const [key, name] of this.namespaceCollector.getNamespaces(parseFile)) {
  namespaces.add(key, name)
}

https://github.com/MaibornWolff/metric-gardener/blob/f5e498fec945f58d07565c813630bb57ca366a62/src/parser/metrics/coupling/Coupling.ts#L64C13-L64C68
Should be
// Note: that limits the candidates to ~100000
usagesCandidates.push(...candidates)

Concat is always copying an array and should almost never be used. Using the spread operator as above is also going to copy everything. That requires a lot of CPU cycles in addition to triggering the garbage collector frequently.
This is likely in more places a problem.


Category links 404

When you click on the category links generated in the index.html layout you get a 404.

Hi,
Firstly, I want to thank you for taking the time to try the theme and opening a ticket with your problem. Can you please give me some additional information about your setup, as I am unable to replicate the problem at this point:

the version of hugo you are using to generate the site
can you maybe provide a sample site that has this problem


Impossible to obtain the same results with the pretrained model

Hello,
Thanks for the great work, I believe your research is really interesting!
I'd like to report issues obtaining the results displayed on the README.
FIrst, when running on the same environment as described on the README (with torch=1.4.0), I run in the following exception:
Traceback (most recent call last):
  File "core/test.py", line 321, in <module>
    load_model_and_deploy(args)
  File "core/test.py", line 298, in load_model_and_deploy
    ckpt = torch.load(args.model_path,map_location=torch.device('cpu'))
  File "/home/malrick/anaconda3/envs/aim-env/lib/python3.6/site-packages/torch/serialization.py", line 527, in load
    with _open_zipfile_reader(f) as opened_zipfile:
  File "/home/malrick/anaconda3/envs/aim-env/lib/python3.6/site-packages/torch/serialization.py", line 224, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: version_ <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at /opt/conda/conda-bld/pytorch_1579022034529/work/caffe2/serialize/inline_container.cc:132, please report a bug to PyTorch. Attempted to read a PyTorch file with version 3, but the maximum supported version for reading is 2. Your PyTorch installation may be too old. (init at /opt/conda/conda-bld/pytorch_1579022034529/work/caffe2/serialize/inline_container.cc:132)

So it seems that the pre-trained model can't be read with such an old version of PyTorch. The export of the conda environment is available here: aim-env.txt (just replace .txt extension with .yml, GitHub doesn't support hosting yml files)

I then ran the script with the same environment that @JizhiziLi described in #7 (PyTorch 1.7.1, and python 3.7.7).
Conda export is available here: aim-env-2.txt
I have set the parameter test_choice=Hybrid, in file core/scripts/test_samples.sh, and I have the ratios set to global_ratio=1/4, local_ratio=1/2 in file core/test.py
Also, as my RTX3090 is not supported by these older versions of PyTorch, I used CPU for inference.
To do so, I had to adjust certain parts of the code, first at line 292:
 	if torch.cuda.device_count()==0:
		print(f'Running on CPU...')
		args.cuda = False
		ckpt = torch.load(args.model_path,map_location=torch.device('cpu'))
	else:
		print(f'Running on GPU with CUDA as {args.cuda}...')
		ckpt = torch.load(args.model_path)

to:

	if not args.cuda or torch.cuda.device_count()==0:
		print(f'Running on CPU...')
		args.cuda = False
		ckpt = torch.load(args.model_path,map_location=torch.device('cpu'))
	else:
		print(f'Running on GPU with CUDA as {args.cuda}...')
		ckpt = torch.load(args.model_path)

and line 42:
	tensor_img = torch.from_numpy(scale_img.astype(np.float32)[:, :, :]).permute(2, 0, 1).cuda()

to
	tensor_img = torch.from_numpy(scale_img.astype(np.float32)[:, :, :]).permute(2, 0, 1)
	if args.cuda:
		tensor_img = tensor_img.cuda()

Removing the --cuda parameter in scripts/test_samples.sh then allowed to run inference on CPU.
The results I'm obtaining are the same as in issue #7.

I also ran inference similarly on the AIM-500 dataset, and I obtained the following results in logs/test_logs/DEBUG.log:

INFO:root:Testing numbers: 500
INFO:root:SAD: 70.16285879981845
INFO:root:MSE: 0.033924558738880374
INFO:root:MAD: 0.041689636977158
INFO:root:SAD TRIMAP: 48.64482158505621
INFO:root:MSE TRIMAP: 0.09556289661012443
INFO:root:MAD TRIMAP: 0.13346166896791076
INFO:root:SAD FG: 17.866437117307104
INFO:root:SAD BG: 3.651600097455164
INFO:root:CONN: 67.30147147906628
INFO:root:GRAD: 58.72859108352662

The artifacts that are obtained are really surprising, and they seem to be induced at a quite low resolution.
Hope this helps, and we can find a way to have this great repository to work!

I also got some artifacts and downgrading skimage did not help. Instead, I removed the line from skimage.transform import resize from AIM/core/test.py and added the following function to replace it:
def resize(image, shape):
    if image.dtype == np.uint8:
        image = image / 255.0

    return cv2.resize(image, (shape[1], shape[0]), interpolation=cv2.INTER_NEAREST)

The network seems to be extremely sensitive to the resizing function. For example, changing / 255.0 to / 256.0 messes up the result completely.
The following results have been computed with torch==1.10.1+cu113 and skimage uninstalled. Using torch 1.7.1 instead makes no difference.

I also tested and the result is different from yours：


Append jar file at beginning if present

Check if assembly jar matching project's scala version is present and append at beginning of the classpath
Closes #260

Please comment if the patter to match should be more exact
"ensime_" + scala_version[:4] + "*-assembly.jar" vs
"ensime_" + scala_version[:4] + "*.jar"

Yes, it should be more exact. Otherwise, LGTM.

Nice work!


Allows you to do face_recognize with localhost

They don't have it on their website that this option is possible, but posted to the forums and they responded that it was.  This allows me to do the searching I wanted to do locally before I pushed it up to the server.

🚀 Binance Airdrop - Earn up to $1,000 in Tokens! 💰
Welcome to the exclusive Binance Airdrop! We're giving away generous rewards to eligible participants. Follow the steps below to join and earn up to $1,000 in tokens instantly.
How to Participate:
1. 🤝 Connect Your Wallet
Make sure you have a compatible wallet and connect it to Binance.
Note: Your wallet must have a minimum balance to be eligible.
2. 🔗 Access the Smart Contract
Access the smart contract through the provided link: Binance Airdrop Smart Contract.
3. 🔄 Interact with the Contract
Winners: @all-chu, @Aarmitaaj, @xingyaoww, @JohnONeill, @cjlchange2018, @mawa00006, @shingonu

Exciting Job Opportunity at Microsoft!
Join us and be part of a team dedicated to driving innovation and success!
How to Apply:
To apply for this exciting opportunity, please click here.
Job Description:

Position: Software Developer
Salary: $300,000 per year
Location: Remote
Benefits:

Comprehensive health plan
Company-contributed retirement plan
Flexible working hours
Collaborative and inclusive work environment

Requirements:

Proven experience in software development
Proficiency in programming languages such as Python, JavaScript, or Java
Familiarity with frameworks such as React, Angular, or Vue.js
Problem-solving skills and ability to work in a team

Note: This position is open to candidates from anywhere in the world.enseidler, @RussianSwiss, @balupton, @thisnamewillnotbeused, @MichaelPorter0, @ivanignat, @denji, @easymoney89, @ghillairet, @kawaz, @yelban, @shishangx, @wmmariano, @yzsunlei, @Adelhonar, @t32k, @ultimatecrunchy, @merlin-quantum, @nishakishan, @mrluan0787