Which is the loss function used for multi-class classification?

Hello!
I'm trying to know which is the loss function used for multi-class classification. I had though it might be the same as in GBM (for K classes):

where y_k=1 if x's label is k and 0 in any other case, and p_k(x) is the softmax function. However, I have made the first and second order gradient using this loss function and the hessian doesn't match the one defined in the code here by a constant 2.
Could you please tell me which is the loss function used?
Thank you!

See https://github.com/dmlc/xgboost/issues/638
I'm also interested in this.

https://github.com/dmlc/xgboost/issues/1825


⚠ Secret Website 3 has degraded performance

In 2e149e2, Secret Website 3 ($SECRET_WEBSITE_3) experienced degraded performance:

HTTP code: 200
Response time: 10292 ms

Resolved: Secret Website 3 performance has improved in 4ff8e29.


CONFIGURATION NOT WORKING

Today the Home Assistant dashboard tells me that the configuration is not working...
Is there anything you know?

Yes, I also get an error message, translated to English it states "Couldn't configure". All sensors are unavailable.
Tried to restart the integration but no luck.
The error might correlate to the last HA update (2023.11)

me too, unfortunately there isnt much information about

I have the same issue since yesterday HA update.

same here.

Same((

@Tasshack, maybe this should be categorized as an issue instead?

Same for me, seems to be in correlation with HA update 2023.11

same here.

Same problem

same here :(

Seems indeed related to HA 2023.11.
Relevant log:
Logger: homeassistant.config_entries
Source: config_entries.py:402
First occurred: 12:35:08 (2 occurrences)
Last logged: 13:08:22

Error setting up entry Cinderella for dreame_vacuum
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dreame_vacuum/__init__.py", line 22, in async_setup_entry
    coordinator = DreameVacuumDataUpdateCoordinator(hass, entry=entry)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dreame_vacuum/coordinator.py", line 125, in __init__
    persistent_notification.EVENT_PERSISTENT_NOTIFICATIONS_UPDATED,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'homeassistant.components.persistent_notification' has no attribute 'EVENT_PERSISTENT_NOTIFICATIONS_UPDATED'


Interactive mode CLI select list does not allow setting color

🎯 Goal: - The CLI interactive mode has multiple select lists (e.g. choose framework), when hovering an item the color is always blue which is not part of our coloring schema. Ideally, it should be white. The vendor lib, ink-select-input, does not allow passing a prop with the color. We should submit a PR ourselves to make 'em support this
🤔 Things to consider:

Consider opening an issue first
Link to repo

For some reason when I'm hovering the items I don't get any color 🤔


Feature/vscode config

Description
This PR adds a VSCode configuration files and accompanying documentation.
Closes #18
Areas of interest for the reviewer
I would like to ask reviewers to clone this repo to their machines, checkout the feature/vscode_config branch and try to setup the VSCode by following the instructions in the README.md.
I would like you to spend some time using this configuration and see, if it is suitable for our use.
If you see that something that should be added/changed/removed, don't hesitate to mention it.
My main goal with this configuration is that provides a good solid editor base for our developers
After-review steps

I will merge PR by myself.

@vid553 @NejcKle @TjazVracko one more thing: before testing this disable settings sync on your VSCode account.

Some more probably usefull settings to add to settings.json:
  "explorer.confirmDelete": false,  // does not ask with popup when you want to delete files
  "window.titleBarStyle": "custom",  // Title bar matches your theme

    "editor.rulers": [100] // under the [c][cpp] section??

  "editor.inlineSuggest.enabled": true,  // This replaces the sugess

// You can use this to control which file types are enabled for copilot
"github.copilot.enable": {
    "*": true,
    "plaintext": false,
    "markdown": true,
    "scminput": false  // This is the git commit message box, haha.
  },

  "diffEditor.ignoreTrimWhitespace": false  // See all changes in whitespace when viewing diffs.

@MarkoSagadin The snippets only work in c files, not in cpp files.
I am not sure how to configure them to work in both. My original setup has this working.

Vscode automatically detects most conf files to be YAML, and then proceeds to show a bunch of yaml sintax errors...
There must be a way to define file associations for conf files, which usually don't have file extension (like <boardname>_defconfig)

I also did a quick check of the nRF DeviceTree extension. It kinda works.
It requires a workspace setting "devicetree.zephyr": "<path to zephyr root>", which I was unable to configure using a relative path. So it would would have to be set for every project specifically.
The extensions shows errors and warning when viewing a DTS file and also support a GUI editor (think CubeMX). But how accurate/valid the information shown actually is I do not know (do not have time currently to check).

@TjazVracko
I force-pushed a commit with requested changes. I added the settings that you suggested except the below ones, I think that they are too specific:
  "window.titleBarStyle": "custom",  // Title bar matches your theme
  "todohighlight.keywords": [
    {
      "text": "TEST:",
      "color": "#ff0000",
      "backgroundColor": "#30CB30",
      "overviewRulerColor": "grey"
    }
  ]

I added instructions for using Profile files. Check, if they are ok.
I also added most of the Nordic plugins except the main one.

@vid553

I had however problems when running extension installation script. The script didn't install any
extensions and threw Buffer is deprecated error on each install attempt.

I have seen this warning on my side as well, but the extensions were installed anyway. This might be
a difference between MacOS and Linux.

the scripts didn't disable / uninstall Microsoft's c/c++ extension, which I think is not desired
to be enabled?

If scripts are used they should be used on a completely clean VSCode setup. They won't uninstall
anything by default. The C/C++ extension is actually disabled in the settings.json:
"C_Cpp.intelliSenseEngine": "disabled", (it must be so that is not fighting with the Clangd).
Did you back up your existing configuration before trying scripts?

I recommend adding the Code Spell Checker extension

We have the tekumara.typos-vscode extension that does the same thing. Benefit of using this one is
that is using the same typos tool that our pre-commit does.

Consider changing Github theme and Noctis extension to be optional as not every user wants to use
any of those themes.

They are only installed but not configured to be used (the used theme is the VSCode default one,
"Default Dark Modern"). Should I remove them from the install_extensions.sh script?

I also think IntelliCode extension should be included as it provides code suggestions for python
and TS/JS projects (writing this as I didn't get any suggestions using the setup that was made
from this PR)

I will add it, I was not aware of it.

I also recommend adding isort extension which automatically sorts the imports for python

So, this extension is supported in Ruff and is also already enabled in our
ruff.toml file that is
provided with the irnas-zephyr-template (see line 5).
To have the consistent formatting in your Python files just copy the ruff.toml into your project's
root dir and it will work.

I also in RN projects use the extension React Native tools, please add it here as well.

I have added it.

Consider adding these settings:

Our Copilot setting already does what you want so that is ok. I will make
"telemetry.telemetryLevel": "none", (thank you for that I was not aware of it).
Everything else I would say that is personal preference, and should be added explicitly by the user.


no ability to reset store

running tests with jest and doing jest.resetModules() before each test results in warning:
[mobx] Warning: there are multiple mobx instances active. This might lead to unexpected results. See https://github.com/mobxjs/mobx/issues/1082 for details.

example code
describe('store:', () => {
  let uut

  beforeEach(() => {
    jest.resetModules()
    uut = require('./store').default
  })

  it('should get event filters', () => {
    expect(uut.getFilters()).toEqual([])
  })

  it('should get events', () => {
    expect(uut.getEvents()).toEqual([])
  })
})

can we have an ability to reset store? or what are our options

The only thing I could come up with to solve this usecase is to deepClone the initialState passed to remx.state(initialState), which incurs runtime costs.
I didn't push this change, as I'm not sure it's a good idea.
The other option is to wrap all your store logic in an object and create a new one each time (for example create it in your module/app's entry point and hold it for the duration of the app (like you would do with redux). For example.


Using expression as number of rows in LIMIT in MATCH

nGQL 1: MATCH (v:player) RETURN v.name AS Name ORDER BY Name | LIMIT (rand32())

nGQL 2: MATCH (v:player) RETURN v.name AS Name ORDER BY Name  LIMIT rand32()

Pipe really matters. But this does not make sense. Please check it out.

The LIMIT clause in nGQL is now only support integer.


Pg/force futures better

Sending out early for review, will add tests soon.

👍
merge when green.
Nice work working through this and measuring the improvement. Can you possibly think about adding a benchmarking system to the build so we can re-run those things (and maybe even thing about how to do a travis check of performance).

Yes, I ultimately plan to add benchmarks here. For now I'm working in our internal repo to refine/grow these benchmarks. We are woefully short of performance testing for online summingbird, want to fix that internally first and then contribute here.


added projects display to dataset admin form

Changes

Added projects display to dataset admin form

Migrations
Screenshots

Checks
Important: please complete these before merging.

[ ] Run migrations, if any.
[ ] Update changelog.md, including migration instructions if any.
[ ] Run unit tests.

Branch is now complete and functional


[Feature Request] Would like to have an eco-mode toggle

The APIs offer an interface to manage the Nest Eco mode. Would be great if toggling heat/off was heat/eco/off or perhaps a config to set off to be eco mode instead of actually off. As a last resort, the other nest plugins expose eco mode as a light switch. I don't love this but something is better than nothing.
I'm willing to help on this but looking for some guidance on how the feature should behave and some pointers on the Apple HomeKit details for what may/may not be possible in terms of replicating the Nest UI.

Duplicate of https://github.com/potmat/homebridge-google-nest-sdm/issues/23#issuecomment-1149955989


SCM message box grows in height when message text input and SCM sidebar visibility toggled to visible using keyboard shortcut

VSCode Version: 1.49.3
OS Version: Windows 10 Pro for Workstations (Windows_NT x64 10.0.19041), macOS Catalina 10.15.7 (Darwin x64 19.6.0), Ubuntu Linux 20.04.1 LTS (Linux x64 5.4.0-48-generic)

I've observed this condition before without realizing the cause, but I finally figured out what's going on and how to reproduce it... If some text is input in the commit message text area and the sidebar is hidden while on the SCM view, then the sidebar is shown again using the View: Toggle Side Bar Visibility keyboard shortcut, the commit message text area is vertically resized inside its container (which is not resized) making the commit message text area look strange and difficult to use properly.
It's easiest to see with 2 or 3 characters input as the commit message. More than that and it can be hard to tell what's going on.
Steps to Reproduce:

Open a Folder with an active SCM registration
Switch to the SCM View
Input 2 or 3 characters in the commit message text area
Hide the Side Bar by either:

Clicking the SCM View Switcher icon
Using the View: Toggle Side Bar Visibility keyboard shortcut

Show the Side Bar using the View: Toggle Side Bar Visibility keyboard shortcut (default: Ctrl+B / Cmd+B)

The issue does NOT occur when showing the SCM view by clicking on its icon when the sidebar is hidden. It also does NOT occur if the sidebar is hidden then shown while on any view other than SCM and switched to SCM view using the keyboard shortcut. It only occurs when using the sidebar shortcut while on the SCM view and a commit message is input.

Does this issue occur when all extensions are disabled?: Yes
Issue occurs on a new/clean installation and also on the current insiders build (1.50.0-insider) with no extensions installed.
Here's a short capture showing the issue where I input "Test" one character at a time and hide/show the sidebar after inputting each character:

BTW, I've seen this issue in earlier versions; it's not caused by something in this specific release.


Can't login without Yammer service enabled

Dear colleagues,
It's not possible to login if you don't have Yammer enabled getting the following error:
AADSTS650052: The app needs access to a service ("https://api.yammer.com/\") that your organization has not subscribed to or enabled. Contact your IT Admin to review the configuration of your service subscriptions.
Thanks and regards,
Ivan

Thank you for reporting the issue and sorry for the trouble. We're considering to do the following:

for the time being, we'll remove the Yammer permission from the CLI AAD app, so that everyone can use it
we'll extend the docs with a remark that to use Yammer commands, you need to use a custom AAD app
we'll research the ability to use dynamic consent with AAD v2 so that everyone can grant additional permissions to the same app without having to provision their own

When using the latest beta of the CLI, you should be able to consent the AAD app in your dev tenant as it no longer refers to Yammer API permissions. If you want to use CLI's Yammer command s in a different tenant, after initially consenting CLI's AAD application, you should run consent --service yammer which will provide you with a URL to grant the AAD application permissions to the Yammer APIs.


[5.7] Don’t call Redis clients directly

To move one step closer to #22961, stop calling PhpRedis/Predis methods directly and pass everything through command(). This is happening for most calls anyhow via Connection::__call().

Hmm, we loose the static analysis capability by refactoring in this way though.

Hmm, we loose the static analysis capability by refactoring in this way though.

It's not a refactoring, it's fixing incongruence and making it more flexible. There are already heaps of calls to command() in PhpRedisConnection.

Can this be re-reviewed @tillkruss. Currently this breaks when using PhpRedis (See #25456).
The $this-client is a \Illuminate\Redis\Connections\Connection and not the Redis client directly. This translates to a \Illuminate\Redis\Connections\PhpRedisConnection with the phpredis driver which does some thing to make sure the parameters are in the correct order for that driver. By calling command directly this is bypassed and never called resulting in a broken application.


[Question] What would be the best way to log user login activity?

What would be the best way to log user login / logout / attempt activity? Thanks for another great package!

For most cases, I'd suggest simply manually calling the activity() function in your controllers where the actions are happening.
Another strategy would be to have your application emit events for login/logout/etc., and setting up a listener to log the actions.
There's not really a generic way-to-go here I think, it depends on your application.

Thanks - I think for the use case of logging Laravel out of the box login specifically, the event approach must be used as the action happens in the framework source as opposed to the Auth controller. Is that correct?

What I did, simply create an Event & a Listener.
Here my LogEvent
<?php

namespace App\Events;

use App\Events\Event;
use Illuminate\Queue\SerializesModels;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;

class LogEvent extends Event
{
    use SerializesModels;

    public $subject;
    public $description;
    public $user;

    /**
     * Create a new event instance.
     *
     * @return void
     */
    public function __construct($user, $subject, $description)
    {
        $this->user = $user;
        $this->subject = $subject;
        $this->description = $description;
    }

    /**
     * Get the channels the event should be broadcast on.
     *
     * @return array
     */
    public function broadcastOn()
    {
        return [];
    }
}

And for my LogEvent listener:
<?php

namespace App\Listeners;

use App\Events\LogEvent;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Spatie\Activitylog\Models\Activity;

class LogEvent
{
    /**
     * Create the event listener.
     *
     * @return void
     */
    public function __construct()
    {
        //
    }

    /**
     * Handle the event.
     *
     * @param  LogEvent  $event
     * @return void
     */
    public function handle(LogEvent $event)
    {
        activity($event->subject)
            ->by($event->user)
            ->log($event->description);
    }
}

So, later on I just call event(new \App\Events\LogEvent(Auth::user(), 'Authentication', 'Successfully logged in.')); to start logging.

Thanks for posting your approach @cleanique-coders
Here's how we do it for our own projects: https://github.com/spatie-custom/blender/blob/master/app/Http/Controllers/Back/AuthController.php#L31

Thanks for the suggestions. I didn't realize that Laravel [broadcasts events for login activity] (https://laravel.com/docs/5.2/authentication#events). So with that known, I ended creating a listener for the Login event and called Activity::log('Login successful'); in the handle method of that listener.


Formatter issues with rust-analyzer.

I used to have BufPreWrite hook with lsp-formatting-sync. That worked great with rls, but after I switched to rust-analyzer it often blocks for minutes before it actually does the formatting. So I decided to change to lsp-formatting. That works great most of the time, but ... if I switch the current buffer after write, the buffer that I've switched to gets overwritten with the reformated code from the previous buffer.
My institution tells me, that if kak-lsp sent a command to kak to switch to the original buffer, replace text, switch back to the current buffer, then everything would work perfectly.

I believe format also fail to work with lastest kak-lsp I tried.

Everything works almost perfectly here. It's just rust-analyzer on every start will start dong the whole indexing from scratch, doesn't do any caching yet, and it takes it a minute or two before it's ready. With -sync it will block the whole kak interface, without it will queue commands and after a minute or two - start rewriting wrong buffers sometimes.

https://github.com/ul/kak-lsp/issues/356#issuecomment-643895885


Enhance Tab replace, Auto-parenthesis for single quote, better {} completion.

The following changes have been brought in this PR :

Added auto-match and complete '. (single quote)
In my last PR #13, I added auto indent on {} was added. However it was limited to C++ and as soon as user types { the closing } was always placed in a new line. Now in this PR, the newline is placed only after user press newline key. This behavior is exactly same as QT Editor's behavior.
Tab Replace when set to false, the indentation was still done using Spaces. Now, when it is set to false the editor places indents with Tabs. This behaviour is good because earlier in Python, lang Mix of spaces and Tabs would produce errors.

@Megaxela Merge soon, I need to update my dependencies to new changes.

Nice work!
I wont complain with code style, cause I'll add clang-format soon.
Thank you so much for you contribution.

I used Google Code style. Yours was similar to Visual Studio. It would be nice if you could include your .clang-format file. So formatting will be okay

Yes, Google Code style is pretty interesting and well documented, but I tried to follow some Qt code style here. I don't know is it documented somewhere, but I'll make some research about it.
It's pretty possible, that I'll include common Google Code style if I wont find any Qt related info.


Email validation too restrictive

The current email regex rejects gmail style email aliases (eg ronco+ember@example.org).
This can definitely be supported using a custom regex, but I wanted to offer up this alias scenario for consideration as I think it's a pretty common use case.
I think you could go much further if interested.  The specific supported syntax for the local-part of email addresses are a bit tough to divine from the various RFCs, but according to RFC3696 the following are also valid characters: ! # $ % & ' * + - / = ?  ^ _  . { | } ~`.
I'd be happy to provide a PR if there's interest.

If it's a valid character in emails, I think we should allow it. I, personally, use this all the time to see who sold my email (e.g. spam to<EMAIL_ADDRESS>If people want to prevent the +, they should make a custom validator IMO.

I agree. Ill add a more extensive email regex.

worth a read: http://davidcel.is/posts/stop-validating-email-addresses-with-regex/

I get the article's point, but when user registration is a key business metric, you really want to make sure that something stupid (like fat-fingering an email address) doesn't cause you to lose a registration. New users are valuable.

I think trying to get a perfect email regex would be a fool's errand.
However there is value in having a 99% solution to help out with fat-fingering and such.  Consumers of this library can always override with their own regex if they want to be vastly more permissive.  I think the current implementation is already covering 98%, it's just us techie types that really make use of the '+' in addresses.

I think what the article really suggests is: be aware of the short-comings, and choose appropriately.


Improve text entry box

Change to multi-line, wrapping textarea, instead of input box... probably as its own component.
https://codepen.io/anon/pen/LaLBmg

#12


Tag 4.0.2 does not exist (anymore?) on docker hub

Thanks for this amazing repo, hoping to leverage it's power during the GoGodotJam tomorrow
On a Gitlab-ci run the runner complains about the missing tag:

See Docker hub tags list

Duplicate of https://github.com/abarichello/godot-ci/issues/98. This probably won't be done by tomorrow, sorry.

Hey that's fine, I plan to use this one instead adrasteondev/godot-ci:4.0.2 not sure if it will work, worst can I'll generate the build manually


Filter dicts have not been implemented in Moto for '%s' yet for instance.group-name

I am attempting to use the describe_instances function in boto3. I am supplying a filter that returns a string, and am receiving the following message:
NotImplementedError: (u"Filter dicts have not been implemented in Moto for '%s' yet. Feel free to open an issue at https://github.com/spulec/moto/issues", u'instance.group-name')
So I am opening an issue.Thank you for your help.

Thanks for opening! I'll add this as a feature request.
There are a lot of feature requests right now though so I can't make any promises on timing.

This was fixed by #1083, so I'll close this


feat config: Allow for JSON/object exports in config files.

Instead of only allowing a function in a javascript file, also accept .json and .js that export objects.
Makes it easier to manipulate karma configuration files with tooling (e.g. yeoman).

/ping @maksimr

@izaakschroeder Thanks!
Please write test on this behavior.
Also we should update documentation that karma can accept json file

ping @izaakschroeder

@Dignifiedquire - I thought I tried looking into the tests for this when I made it and was unsure how to approach it. Direction welcome.

Sure @izaakschroeder take a look at this unit test for the config https://github.com/karma-runner/karma/blob/master/test/unit/config.spec.coffee you should be able to add a new test case in there where you pass in a config which contains a JSON object and make sure it parses as expected.

@izaakschroeder ping

@izaakschroeder any news?

Closing due to inactivity


Fix KNXClimate mode change not updating the UI

Description:
Fix KNXClimate not updating the UI after receiving mode change telegram from KNX bus.
For example: If a mode change event was received (STANDBY, COMFORT, etc.) the UI component was not updated. Component only got updated when temperature change was received.
Checklist:

[ x ] The code change is tested and works locally.
[ x ] Local tests pass with tox. Your PR cannot be merged unless tests pass
[ x ] There is no commented out code in this PR.
[ x ] I have followed the [development checklist][dev-checklist]

Hi @tombbo,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!


指定流量使用上游代理

我有一台韩国VPS,并成功搭建了代理，韩国也有墙，不能访问某些小姐姐网站。怎么把只访问小姐姐的流量转发到香港VPS上处理呢？

Hi there,
I have plugged this into Google Translate because I don't know Chinese. For convenience of others, here is the translation:

Specify upstream traffic using upstream proxy
I have a South Korean VPS and have successfully built an agent. South Korea also has a wall and cannot visit certain Miss Sister websites. How do you forward the traffic to the Missy Sister to the Hong Kong VPS?

@allenchennnn this is a forum for developers, please seek support at https://caddy.community
To ensure people understand what exactly you are asking, spell it out extremely verbosely, and include translation by Google Translate.
If I did understand you correctly, It sounds like it's related to upstream proxy that is a work in progress #27. However, upstream proxy will not have functionality to limit what is being upstreamed: it is either all traffic or nothing.


Add info about creating local Dream dependency from external dir

Add information about how to configure projects that need to reference
the local version of Dream from inside an external directory.

Thanks!


feat(report): add hint link to result HTML

To follow up #685 , this pull request adds a hint link (❔) to the upper right of the test result diff table (example). Not a beautiful one, but I hope it works.

Thanks @galtm for the review.
I'm merging this. The unfinished linked page will be tracked in #685.


Issues with fetchSockets()

Hey there,
we are currently facing multiple issues with fetchSockets() in a Nest.js backend using @socket.io/redis-adapter.
First, we tried to use server.to("room").fetchSockets().then((socket) => ...) to emit to all sockets across all containers (with the help of Redis adapter) as we needed to process every socket individually (emitting user-specific data to each socket).
We are running on AWS ElastiCache and noticed that when using the aforementioned mechanism, only sockets of the local container are returned, thus nobody who is connected to other containers will receive the event. This works fine locally across multiple containers though.
We came up with a work around that uses server.local.to("room").fetchSockets() and then emits a server-side event in order to do the same on all other containers. We also found this to scale better as we're distributing business logic that runs on those sockets to all containers. So while we were pretty happy with the work around, it doesn't work unfortunately. And this time it is locally reproducible. For some reason, server.local.to("room").fetchSockets() does (inconsistently) return all sockets across all containers, thus emitting the event to every socket multiple times.
When subscribing to Redis Pub/Sub using redis-cli it also looks like every container is requesting all sockets from all containers, although using server.local.....
We are pretty lost currently when it comes to ideas where to debug and how to fix this issue, therefore we're asking here. We also asked on the Nest.js discord, although we believe this is either a problem with the Redis adapter or with socket.io itself (thus my next question: does this issue belong in this repository or should we open another issue in the socket.io repo?).
Thanks a lot for your help!
Relevant versions in use

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Reproduction code

server: https://github.com/fan711/socket-server
client: https://github.com/fan711/socket-client

/cc @fan711

I might have pinned the issue down to be in BroadcastOperator thus not being an issue of the Redis adapter. Opened an issue in the socket.io repository: https://github.com/socketio/socket.io/issues/4359

You are absolutely right, the flags was not forwarded to the adapter.
This should be fixed by https://github.com/socketio/socket.io/commit/30430f0985f8e7c49394543d4c84913b6a15df60.
Thanks a lot for the detailed report :+1:

This was fixed in<EMAIL_ADDRESS>https://github.com/socketio/socket.io/releases/tag/4.5.1


Can you explain the difference between the master branch and the feat807-i18n branch?

Can you explain the difference between the master branch and the feat807-i18n branch?

is there any difference in function?
will you release the Chinese version?
is the branch feat807-i18n stable?
Thank you

It's a Work in progress
The related PR is here : https://github.com/portainer/portainer/pull/1134

The master branch is the stable version of Portainer.
As @WTFKr0 specified, the feat807-i18n is currently marked as a work-in-progress, meaning it's not fully functional yet.
It contains some work on the internationalization of Portainer but unfortunately I'm working on other priorities right now so I cannot work on it. Also, this task is really time consuming as it requires to insert placeholders for translations in all the HTML files (and that's only for the front-end).
I'd love to see Portainer translated in multiple languages and I believe this will be possible once the implementation in #1134 is done.
I'd also accept contributions based on the changes I've started to work on in #1134

Thank u

@WTFKr0 @deviantony
Are you abandoning the plan for internationalization now?
Because I found that the recent submission of feat807-i18n was a year ago.

Hi @daiaji
It's not abandonned, it's just not in our priority list at the moment. Work on feat807-i18n was originally done by a contributor, we do accept contributions on this topic.


Is mediaconvert good option for video editing?

Mediaconvert is good for encoding, but we want something that can cut the video or merge the video. We know MC gives you that option, but looks expensive.
Is it better to build in house video edit software or chose other vendor?

Thanks for your comment. AWS Elemental MediaConvert supports both Input Stitching and Clipping (more info here: https://docs.aws.amazon.com/mediaconvert/latest/ug/assembling-multiple-inputs-and-input-clips.html).


Fix issue with blank pod name/ns

Pod Name / Namespace was blank in some scenarios.
Description
Minor fix to fill in empty pod name with pod generatename and pod namespace from request namespace if empty.
Both these fields were empty in my EKS 1.22 cluster (and so no instrumentation rules would match).
Type of Change

[X] Bug Fix
[ ] New Feature
[ ] Breaking Change
[ ] Refactor
[ ] Documentation
[ ] Other (please describe)

Hi, could you share your use case? I'm always interested in scenarios this instrumentation tool helps...

@chrlic Our usage scenario was similar to the blog post. We use a CI/CD pipeline with terraform/helm to automate deployments. We found that whenever we did a helm upgrade (just to change a values.yaml parameter) all the workloads would get restarted due to the statefulset/deployment being patched by helm (to remove the appd additions) and then the AppD cluster-agent would modify the statefulset/deployments to add AppD back.
In my env the AdmissionReview (admission.k8s.io/v1) request JSON always had .request.object.metadata.namespace missing, only .request.namespace had a value.
AWS EKS v1.22.17-eks-48e63af
Rancher v2.6.8


Could not find JS function on MacOSX Big Sur

Checklist

[ x] I'm reporting a broken site support issue
[x ] I've verified that I'm running youtube-dlc version 2020.10.31
[ x] I've checked that all provided URLs are alive and playable in a browser
[x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x ] I've searched the bugtracker for similar bug reports including closed ones
[x ] I've read bugs section in FAQ

Verbose log

frog@My-MacBook-Pro wild_kratts % ../youtube-dlc -v 'https://www.youtube.com/watch?v=iyAefH3Bw1c'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=iyAefH3Bw1c']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.11.11-3
[debug] Python version 2.7.16 (CPython) - Darwin-20.6.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] iyAefH3Bw1c: Downloading webpage
[youtube] {18} signature length 104, html5 player f3c4e04d
[youtube] iyAefH3Bw1c: Downloading player https://www.youtube.com/s/player/f3c4e04d/player_ias.vflset/en_US/base.js
ERROR: Signature extraction failed: Traceback (most recent call last):
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
    video_id, player_url, s
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "../youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
 (caused by ExtractorError(u"Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.",)); please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
    video_id, player_url, s
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "../youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "../youtube-dlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
    ie_result = ie.extract(url)
  File "../youtube-dlc/youtube_dlc/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 2202, in _real_extract
    encrypted_sig, video_id, player_url, age_gate)
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1405, in _decrypt_signature
    'Signature extraction failed: ' + tb, cause=e)
ExtractorError: Signature extraction failed: Traceback (most recent call last):
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
    video_id, player_url, s
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "../youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "../youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
 (caused by ExtractorError(u"Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.",)); please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.

frog@My-MacBook-Pro wild_kratts % ../youtube-dlc --version
2020.11.11-3
tim@Tims-MacBook-Pro wild_kratts % sw_vers
ProductName:	macOS
ProductVersion:	11.6
BuildVersion:	20G165

Description

File failed to download due to Could Not find JS function error.

I am getting the same JS function error on macOS 12.1
youtube-dlc version 2020.11.11-3
######## Verbose bug info below
bash-3.2# youtube-dlc -x --audio-format mp3 https://youtu.be/UXUwLyX6sUg
[youtube] UXUwLyX6sUg: Downloading webpage
[youtube] UXUwLyX6sUg: Downloading js player f3c4e04d
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
(caused by ExtractorError(u"Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.",)); please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
bash-3.2# youtube-dlc -v -x --audio-format mp3 https://youtu.be/UXUwLyX6sUg
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-x', u'--audio-format', u'mp3', u'https://youtu.be/UXUwLyX6sUg']
[debug] Loading archive file None
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.11.11-3
[debug] Python version 2.7.18 (CPython) - Darwin-21.2.0-arm64-arm-64bit
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2
[debug] Proxy map: {}
[youtube] UXUwLyX6sUg: Downloading webpage
[youtube] {18} signature length 104, html5 player f3c4e04d
[youtube] UXUwLyX6sUg: Downloading player https://www.youtube.com/s/player/f3c4e04d/player_ias.vflset/en_US/base.js
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
(caused by ExtractorError(u"Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.",)); please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 830, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 532, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 2202, in _real_extract
encrypted_sig, video_id, player_url, age_gate)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1405, in _decrypt_signature
'Signature extraction failed: ' + tb, cause=e)
ExtractorError: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1395, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1295, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 1370, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dlc/youtube_dlc/jsinterp.py", line 245, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
(caused by ExtractorError(u"Could not find JS function u'na'; please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.",)); please report this issue on https://github.com/blackjack4494/yt-dlc . Make sure you are using the latest version; type  youtube-dlc -U  to update. Be sure to call youtube-dlc with the --verbose flag and include its complete output.
bct@2021 bashScripts % sw_vers
ProductName:	macOS
ProductVersion:	12.1
BuildVersion:	21C52

I switched to yt-dlp and got things working again.
Install and demo workflow linked here: https://youtu.be/dAc5VIDIs5k


mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))

Using Python Version 3.10.8
System Version:	macOS 12.6.1
Kernel Version:	Darwin 21.6.0
Chip:	Apple M1 Pro(arm)
Name: snappi
Version: 0.9.5
Python 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-14<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import snappi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/--/Applications/pyenv/lib/python3.10/site-packages/snappi/__init__.py", line 1, in <module>
    from .snappi import Config
  File "/Users/--/Applications/pyenv/lib/python3.10/site-packages/snappi/snappi.py", line 13, in <module>
    import grpc
  File "/Users/--/Applications/pyenv/lib/python3.10/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/Users/--/Applications/pyenv/lib/python3.10/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/Users/--/Applications/pyenv/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): tried: '/Users/--/Applications/pyenv/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))

@Deepankarkr could you please try installing grpc libraries separately and let us know if it works ?
python -m pip install grpcio
python -m pip install grpcio-tools

@Deepankarkr kindly reopen if you still see this issue.


Collective Navigation – Elements and buttons polish

We are adding the final input on the UI elements of the navigation component:
Button states:

Update category name in mobile while scrolling:
We currently have a specific behavior to indicate users where they are when scrolling the page

This is the way we do the same on smaller screens

See the 🎨 design file in figma
@Betree

This has been taken care of in the above PR except for the "Categories menu elements" states. Is this a priority @Betree?


Stack should either loosen or tighten constraints, not pass them through

Today, Stack passes its constraints through to its children, which has confused a number of people.  Instead, the Stack should loosen its constraints by default but probably have an option to tighten them instead.
We might want to change the default alignment for stack at the same time.  Currently we default to a top-left alignment but we should probably default to a center alignment.
/cc @Hixie @apwilson

Does this need to be included in api stability?

Yes.  We won't be able to change the layout rules for Stack once we declare API stability.

I tried doing this, but ran into https://github.com/flutter/flutter/issues/9408.


feature: synchronize topic-config name to be in sync with topic-name (add checkbox to enable)

after second thought this enancement is not needed at all


We are preparing your Mac...

Would anyone have an idea why I might be stuck at the "We are preparing your Mac..." stage of SplashBuddy? I have removed all but one app, re-imaged multiple time, deleted the computer from JAMF, and I can't see to get passed this stage. The applications to get installed in the background from JAMF because of the "on enrollment" trigger, but SplashBuddy is not seeing it for some reason.
I'm testing this on 10.13.5 running with JAMF Pro 10.5.

make sure the name of the .pkg file maches the packageName in the preferences file. You can find the exact package name by doing cat /var/log/jamf.log | grep .pkg and look for the package that matches your application you are tracking for installation.
Ex: For a OneDrive package named OneDrive-1.203.455
Your plist might look like something like this -
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>applicationSideMenu</key>
	<true/>
	<key>continueAction</key>
	<string>Restart</string>
	<key>applicationsArray</key>
	<array>
                <dict>
			<key>canContinue</key>
			<true/>
			<key>description</key>
			<string>VPN Client</string>
			<key>displayName</key>
			<string>Pulse Secure</string>
			<key>iconRelativePath</key>
			<string>pulseSecure.png</string>
			<key>packageName</key>
			<string>PulseSecure</string>
		</dict>
        </array>
	<key>assetPath</key>
	<string>/Library/Application Support/SplashBuddy</string>
	<key>htmlPath</key>
	<string>presentation.html</string>
	<key>webRefreshRate</key>
	<integer>60</integer>
</dict>
</plist>

One way I test is make the item available in self service, install from self service. The once I'e built my SB and set the prefs install it on the same Mac and if all correct it will go green as SB will pick it up from the JAMF log.

Thanks for the help guys! @cybertunnel was right, I just didn't have the naming correct. Working like a charm now!


Corrections on DO_SET_SERVO mission item

Hello,
I have modified default auxiliary mixer to accept input from actuators group 2
Next i added bootlog.txt as output into rc.interface to have more visible mixers loading process when user try to upload own mixer
Last change - adding DO_SET_SERVO mission item check in navigator( before only first mission item are checked) and correction of limits for id of AUX outputs ( from 0...5 to 1...6)
Changes are tested in flight today for all 6 AUX channels.
Regards,
Michail.

Thanks for the update!

Hello, Lorenz,
I have removed double check and tested yesterday in flight.
Tested for normal work and for critical situations when i try to enter wrong values, when i put this mission item at the start or end of mission.
Regards,
Michail.

Hello? Lorenz,
Do you need some additional actions from my side, if not - when you expect to push it to master?
Michail.

Lorenz is currently travelling, he'll be back by the end of the month

Hi,
Thanks for this information, will wait.
Regards.

Hello, Lorenz!
Can you have a look to this PR?
Thanks in advance.
Michail.

Navigator change looks good, the mixer change is difficult - you can now have conflicting inputs on the payload.
I would like to understand the issue you were trying to solve - is the issue that manual pass-through in your case is not a good thing? It seems like we would need a payload controller arbitrating between manual and automatic passthrough mode.

@thedevleon I think you also fixed the pass-through mixer in your PR. I'd like to get that onto master before the release.

Hi!
Finally i search to manipulate payload in mission, and you propose to use other subsrciptions? I do not understand where we can have a conflict.
My idea of mixers modification is following - you have a camera shooter on copter. When you do preflight checks you can check functionnement of shooter with AUX switch on RC radio, when you fly the mission to made an aerial map - you use mission items 183 to made shoot a pictures.
But always users can load other mixers if it will be some specific airframes or configuratoins.
Regards,
Michail.

Ok, i see - it can confict with trigger module, i never checked with control group this module use....

Can one of the admins verify this patch?

@fpvaspassion status?


Remove sessions caches from KCB when Persistent Sessions is enabled

Closes #953

Depends on keycloak/keycloak#32416

"depends" is a strong word, it will just print warnings about the caches not being found. Those are from the lb-check endpoint.

it will just print warnings about the caches not being found

Will the lb-check fail of they don't exist? Or should I rather wait for the PR to be merged first?

When the cache does not exist, it skips the cache. The cache is always UP.

Updated build.sh to generate the example for the Keycloak docs.

I've updated the Infinispan version and dealt with the nonexisting caches.


load_model fails on saved model from 09. Milestone Project 2

recreate:
!wget https://storage.googleapis.com/ztm_tf_course/skimlit/skimlit_tribrid_model.zip
!mkdir skimlit_gs_model
!unzip skimlit_tribrid_model.zip -d skimlit_gs_model
loaded_model = tf.keras.models.load_model('skimlit_gs_model/skimlit_tribrid_model/')

AssertionError                            Traceback (most recent call last)
[<ipython-input-114-204ece70eaf7>](https://localhost:8080/#) in <module>
----> 1 loaded_model = tf.keras.models.load_model('skimlit_gs_model/skimlit_tribrid_model/')
      2 loaded_model.summary()

1 frames
[/usr/local/lib/python3.8/dist-packages/tensorflow/python/checkpoint/checkpoint.py](https://localhost:8080/#) in assert_existing_objects_matched(self)
    850       # Display max number of 10 variables in error message.
    851       num_variables_to_show = min(10, num_unused_python_objects)
--> 852       raise AssertionError(
    853           f"Found {num_unused_python_objects} Python objects that were "
    854           "not bound to checkpointed values, likely due to changes in the "

AssertionError: Found 1 Python objects that were not bound to checkpointed values, likely due to changes in the Python program. Showing 1 of 1 unmatched objects: [<tensorflow.python.ops.lookup_ops.MutableHashTable object at 0x7f523de00c40>]

Hi @wallaceturner ,
Thank you for this.
Looks like it was an error with the original saved model and an older version of TensorFlow.
Have fixed this in an update and the above code should work now:
# Download pretrained model from Google Storage
!wget https://storage.googleapis.com/ztm_tf_course/skimlit/skimlit_tribrid_model.zip
!mkdir skimlit_gs_model
!unzip skimlit_tribrid_model.zip -d skimlit_gs_model

See the full (updated May 2023) notebook: https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/09_SkimLit_nlp_milestone_project_2.ipynb


Fix broken link

Codecov Report

Merging #136 into master will increase coverage by 0.18%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #136      +/-   ##
==========================================
+ Coverage   65.04%   65.23%   +0.18%
==========================================
  Files          16       16
  Lines         535      535
  Branches       34       31       -3
==========================================
+ Hits          348      349       +1
+ Misses        187      186       -1

Impacted Files
Coverage Δ

.../cakesolutions/kafka/akka/KafkaConsumerActor.scala
58.48% <0%> (+0.44%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d8eeaf...cec7308. Read the comment docs.


feat: Zenoss event handler

This PR adds Zenoss event handler as requested by https://github.com/influxdata/feature-requests/issues/72. Fixes https://github.com/influxdata/kapacitor/issues/2483.
The handler provides options for Zenoss basic event fields:

handler option
event field
default value

summary
Summary
'{{ .Message }}'

device
Device
"" (empty string)

component
Component
"" (empty string)

severity
Severity
severity level (see Severity mapping)

eventClass
Event Class
"" (empty string)

eventClassKey
Event Class Key
"" (empty string)

collector
Collector (optional)
none

message
Message (optional)
none

In addition to the basic fields, custom fields can be set with customField option. The value of a custom field can be  simple type or a string representation of JSON object. There are also handler options to override Zenoss event-related values set in the configuration: action, method, type, tid. All these options support templates with the following variables available: Name, TaskName, ID, Message, Details, Time, Duration, Level, Recoverable and Tags.
Severity mapping
Alert level is mapped to Zenoss event severity according to mapping specified in the configuration file. Severity can be specified either as a number or a string (see Event severity levels). The default configuration maps OK to Clear,  Info to Info, Warning to Warning and Critical to Critical.
TICKscript examples:
stream
  |from()
    .measurement('cpu')
  |alert()
    .crit(lambda: "usage_user" > 90)
    .message('Hey, check your CPU')
    .zenoss()
        .device('{{ index .Tags "host" }}')
        .component('CPU')
        .eventClass('/App')

stream
  |from()
    .measurement('cpu')
  |alert()
    .crit(lambda: "usage_user" > 90)
    .message('Hey, check your CPU')
    .zenoss()
        .action('CustomRouter')
        .method('add_event')
        .device('{{ index .Tags "host" }}')
        .component('CPU')
        .eventClass('/App')
        .collector('localhost')
        .customField('ticks', 3300)

Default configuration:
[zenoss]
  # Configure Zenoss.
  enabled = false
  # The Zenoss URL.
  url = "https://tenant.zenoss.io/zport/dmd/evconsole_router"
  # Username for HTTP BASIC authentication
  # username = ""
  # Password for HTTP BASIC authentication
  # password = ""
  # Action (router name).
  action = "EventsRouter"
  # Router method.
  method = "add_event"
  # Event type.
  type = "rpc"
  # Temporary request transaction ID.
  tid = 1
  # Alert level to event severity mapping.
  severity-map = { OK = "Clear", Info = "Info", Warning = "Warning", Critical = "Critical" }

Required for all non-trivial PRs

[x] Rebased/mergable
[x] Tests pass
[ ] CHANGELOG.md updated

Can you move this to try to merge with v1.6-master instead of pointing at current master?

@alespour :Can you move this to try to merge with v1.6-master instead of pointing at current master?

@docmerlin Done.


JavaScript

JavaScript
User Story
As a boot camp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
Acceptance Criteria
GIVEN a Prework Study Guide website
WHEN I view the study guide
THEN I can see the four topics I learned listed along with a suggestion on what I should study first

Used shortened versions of ES6 where possible


Feature parity between the plugins/ bind mount and PLUGINS environment variable

Enhancement Type
Improve an existing feature
Describe the enhancement
I've been using the PLUGINS enviroment variable to deploy "packages" of plugins to my servers, but the PLUGINS feature only copies .jar files instead of copying all files like the /plugins bind mount. I was wondering if we could get an option to have feature parity between the two so I can deploy and interpolate plugin configuration along with the jar files in packages.
I'm doing things this way as some of my servers require different "packages" of plugins, eg, a creative world not requiring survival-orientated plugins, but multiple survival worlds all requiring the same survival plugin config.

To make sure I understand correctly, you would like to have the synchronization and interpolation features of the "/plugins" mechanism, but with the ability to source from several directories instead of just the one path?

Yeah, that's basically what I'd like to be able to do!


endpoint: Always compute proxy policies after DNS proxy updates

The planned logic was to only update proxy network policy if the added policy map entries included proxy redirections. This design did not consider the possibility a concurrent endpoint policy computation consuming the part of the policy map entries with redirection, which lead to the policy update to be missed after the remaining FQDN selectors were applied.
For example:

DNS proxy answer to query 'www.example.com' applies to two FQDN selectors: 'www.example.com' with an L3-only rule, and '*.example.com' with a rule on port 80 for which HTTP policy only allowing GET requests is applied.
New local security ID is allocated for the IP in the DNS answer
3. Endpoint regenerations are spawned due to the new ID (which could also match an CIDR-based identity selector in addition to the FQDN selectors).
DNS proxy adds the new security ID to the '.example.com' selector associated with the HTTP rule. 5. Endpoint regeneration consumes the above ID update (from another goroutine). As this update is associated with an HTTP rule, proxy network plicy is recomputed. During this computation the new security ID is only associated with selector '.example.com'.
DNS proxy adds the same new security ID to the selector 'www.example.com', which has no HTTP rules.
DNS proxy waits for the FQDN selector updates to finish.
DNS proxy consumes the computed policy map keys, but only sees the ones for the selector 'www.example.com' as the ones for the selector `*.example.com' were already consumed. Since 'www.selector.com' has no HTTP rules, the proxy network policy update is skipped.
DNS proxy responds to the pod with the DNS answer.
pod sends a HTTP POST request to the resolved IP on port 80.
Traffic to port 80 is redirected to the proxy.
Proxy only has the ID associated with the destination IP in the rule that only allows PUT requests and drops the request.
Proxy policy is synced by a controller and the proxy policy is healed, so that it is eventyally consistent. This was too late for the traffic the pod was sending.

Fix this by unconditionally computing proxy policy at (8) above.
A more efficient implementation would make sure that all the map updates due to a specific DNS response get to each endpoint's policy map changes as one transaction, so that it is only possible to consume them as one unit. This would make sure that proxy network policy updates are done for all of the updated selectors if any of the them have proxy redirections.
DNS proxy now always updates the proxy policy to avoid intermittent policy drops.

/test

Removed dead code.

/test

/ci-e2e


OpenApi models generation

Hello,
I am new to swagger/openapi and trying to generate models based on OpenApi specifications for Azure DevOps, specifically for build.json ver 5.0.
I see that version referenced is "swagger": "2.0". But trying to generate models out of this specification leads to different errors depending on swagger version used.
Latest stable codegen 2.3.1 ignores all the additional properties of object if it has allOf in it.
For example BuildDefinition object:
  "BuildDefinition": {
      "description": "Represents a build definition.",
      "type": "object",
      "allOf": [
        {
          "$ref": "#/definitions/BuildDefinitionReference"
        }
      ],
      "properties": {
        "badgeEnabled": {
          "description": "Indicates whether badges are enabled for this definition.",
          "type": "boolean"
        },
        "buildNumberFormat": {
          "description": "The build number format.",
          "type": "string"
        },
        "comment": {
          "description": "A save-time comment for the definition.",
          "type": "string"
        },
        "demands": {
          "type": "array",
          "items": {
            "$ref": "#/definitions/Demand"
          }
        },
        "description": {
          "description": "The description.",
          "type": "string"
        },
............

Generation command:
java -jar swagger-codegen-cli.jar generate \
   -i build.json \
   -l csharp \
   -o test \
   -Dmodels

It generates class with all the properties of BuildDefinitionReference, but not a single "native" property like BuildNumberFormat and so on.
Am I on wrong version, or missing some arguments? Which tool is best to use to generate models according to these specs?

Thankful for any hints/suggestions!

I am having problems generating code as well. Neither swagger-codegen-cli nor autorest are able to produce C# code that actually works. I haven't been able to find documentation regarding use of these swagger definitions. Some help would be greatly appreciated.

@dnws / @kjartab , the only code generation tooling I am familiar with is autorest.  I do know that some people have had success with it.  The swagger 2.0 files in this repo are used as the only source to generate our REST API documentation, and that is their main purpose.  Swagger 2.0 has limitations, and you will see that we have to leverage a number of custom properties to fully describe our APIs.  Generic code generators are not going to know about the custom fields.  All of our Swagger files should match the Swagger 2.0 specs.  If you find any where that is not the case, please let me know.  When choosing a code generator ensure that that generator has full support for Swagger 2.0.
Overall my recommendation is to leverage one of our existing client libraries, rather than generating one yourself.  These client libraries provide a lot of benefits that you will not get from code generated using the swagger as the source.
Thanks,
Ted

@tedchamb Thanks for the quick reply. I have tried the client libraries you refer to and they work great, but as I am trying to use these APIs in an azure function v2 context combined with cosmos db and azure servicebus, it's not really an option. Long story short; functions v2 has to be used, meaning .net core and therefore it doesnt support .net framework dependencies (which, sadly, is needed for a tiny subset of the dependencies of these client librares). Any plans for converting them to .net standard 2.0? I will try a bit more with the generators then.

@kjartab, all of our preview NuGet packages include NetStandard 2.0 versions of the binaries.

You will have to excuse my lacking understanding of how .net standard 2.0/core 2.x/framework 4.6.x packages work, but the messages in the image above keep popping up. Microsoft.AspNet.WebApi.Client can be upgraded to 5.2.7 which then is restored to .net standard and the issue is resolved. But Windows.Bcl.Build is stuck on .net framework, so when this is loaded into an azure function v2 (.net core 2), which could be running on linux for that matter - it wont work. As I understand it that is, but I would like it if thats wrong.

@kjartab ,
Unfortunately we have some dependencies that do not have specific binaries built for NetStandard for either the version we need and in some cases even for the latest version.  The good news is that portable binaries are fully compatible with NetStandard, and they do supply portable versions of their binaries in their NuGet packages.  You just need to add a fallback in your project file, so it knows which one to prefer.
  <PropertyGroup>
    <AssetTargetFallback>$(AssetTargetFallback);portable-net451+win8</AssetTargetFallback>
  </PropertyGroup>

@tedchamb Now I am a bit confused as it seems to just work now with .net.sdk.functions 1.0.24 og az func v2. In any case: Referencing these client libraries in a .net standard 2.0 project and then using them in an azure function v2 app works fine. Thank you for your help!


Add yq to kong-dev-tools

Pull Request Template
📲 What
Add the yq tool to the kong-dev-tools image
🤔 Why
Often when managing Deck (Kong's CLI tool) YAML files, you need to perform additional yaml modifications and the yq tool is what Kong recommend is used.
🛠 How
Add the yq tool to kong-dev-tools docker image

Used powershell instead so no longer needed


Limit transactions size

Summary
This pull request fixes a bug where inventory transactions were being processed incorrectly when the quantity of inventory actions was too large.
Details
When the quantity of actions in an inventory transaction exceeded a certain value, the server was freezing.
This pull request fixes the issue by adding a check to ensure that the quantity of actions does not exceed the maximum allowed value before processing the transaction.
Impacted Areas in Application

Inventory transactions

I don't think this is the right solution to this problem. I think we should approach the problem from much lower level in gophertunnel. I also don't see exactly how the server would be freezing, given it would at max block just one goroutine.

Yes, I did not explain myself well, it would indeed block that a goroutine but it would nevertheless use resources for nothing.


is loading full models file even necessary?

It seems that I'm only using the stripped file... which is also much smaller

better safe than sorry.. if it is just a matter of disk space, better to keep full models around


Validation of Physical Layer Model with simulations

To validate the Physical Layer PSF model, we will use simulations that include a Zernike prior.  Metrics and plots should be generated and compared with those reported in Tobias' thesis (section 6).
The simulations are located on Jean-Zay: /gpfswork/rech/ynx/commun/data/euclid_sims.

Check out a new branch from case_study_psf_decontamination to use for validation
Update the configuration files: data_config.yaml, training_config.yaml, etc. with the configuration settings described in Tobias' thesis
Run WaveDiff for each dataset
Produce similar plots as found in the Chapter 6: Results section
Evaluate whether the results resemble those in the thesis

What does 'SFE' stand for in the name of datasets, and are the 'nm' numbers for SEDs resolution?

The answers to your questions are explained in section 6 of Tobias' thesis.

Thank you, Jennifer. I am reading Chapter 6 from the beginning so I don't miss anything. I have some questions.

Tobias mentioned in his thesis that he used only the dataset with SFE. Why do we need the one with no_SFE?
What is the number of stars in these simulated datasets?
Are the plots to be produced only for the waveDiff-polygraph model?
Why do we have a different number of Zernikes in the training configuration (15) compared to the metrics configuration (45)?
The same goes for the number of bins: 8 for training and 20 in metrics.
What does the parameter M=64 mentioned in the thesis refer to?

Hi Nada, it’s good that you read chapter 6 and aiming to develop some
understanding surrounding this task.
As I believe one of your goals is to do research, i want to remind you that
a fundamental part of being a researcher is “Reasoning”.
So, with your questions first try to determine the answer yourself. To aide
you, you can look at other parts of Tobias’ thesis by doing keyword
searches. For some questions the answer may be obvious, whereas for others
your reasoning may come with some uncertainty. It is those questions where
you think your reasoning is uncertain that you should ask for clarification.
The point is don’t just ask what is something or why. Try to determine this
yourself and present what you think along with your question.
I will answer one of your questions directly.
You need to run WaveDiff on the model with the physical layer. You will see
which one it is in the e training config file in the case study branch.
On Mon, 8 Apr 2024 at 11:25, nadamoukaddem @.***> wrote:

Thank you, Jennifer. I am reading Chapter 6 from the beginning so I don't
miss anything. I have some questions.

Tobias mentioned in his thesis that he used only the dataset with
SFE. Why do we need the one with no_SFE?
What is the number of stars in these simulated datasets?
Are the plots to be produced only for the waveDiff-polygraph model?
Why do we have a different number of Zernikes in the training
configuration (15) compared to the metrics configuration (45)?
The same goes for the number of bins: 8 for training and 20 in
metrics.
What does the parameter M=64 mentioned in the thesis refer to?

—
Reply to this email directly, view it on GitHub
https://github.com/CosmoStat/wf-psf/issues/133#issuecomment-2042276073,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXRVNTAAC35ASPNMHMUKGLY4JO7LAVCNFSM6AAAAABFWFMZ2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBSGI3TMMBXGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

I noticed the M parameter in the thesis, but it's not in the configuration file, so I was wondering why. You mentioned comparing the plots to those in the thesis, which were done with the dataset with SFE. When I apply it to the no_SFE dataset, what should I compare it to? Additionally, Tobias used different models of WaveDiff in his thesis, but the configuration files only mention implementation for the wavediff polygraph. Perhaps in older versions of WaveDiff, the other models were implemented but I've never worked with older versions. I'm asking questions because we don't have much time for this case. Could you please provide a timeline for finishing this task?

Have you launched any runs at all? Before asking you to do this task, I tested that running was possible for one of the datasets. So, it should be possible to do so even with gaps in understanding.
WaveDiff v2.0 does not include these additional models besides: Polychromatic model ("poly") and now this Polychromatic with the Physical Layer PSF ("physical_poly") models. As I wrote in the description and as we discussed in the past PSF meetings, the goal of this task is to validate the new PSF model with the physical layer.
The different options: "complete", "parametric", and "non-parametric" are set with this option in the configuration file.  While I have only tested with the "complete" setting
@tobias-liaudat is going to share the private repository which you can fork.  It contains a set of notebooks you can use to regenerate the plots with the new PSF model.  Note there may be other adaptations you have to do besides adjusting the paths to the results.

I launched the training part, but I set the batch size to 16 so that I don't encounter the ResourceExhausted error.

@nadamoukaddem can you share the error in your log here?

This is the metrics configuration file: metrics_config.log
and this is the ouptut: metrics_05_seed1.log

Hi @nadamoukaddem , the problem is that PSF model weights file is not found in the trained_model_path you provided.  I copied a portion of the log below, where the code prints the path from where it will attempt to load the psf_model weights.
2024-04-15 11:01:55,005 - wf_psf.utils.read_config - INFO - Loading.../gpfswork/rech/ynx/uch76qv/tests/outputs/wf-outputs/wf-outputs-SFE-05nm-seed1/c$
2024-04-15 11:01:55,048 - wavediff - INFO - <wf_psf.utils.configs_handler.MetricsConfigHandler object at 0x1522d5adb070>
2024-04-15 11:01:55,066 - wf_psf.psf_models.psf_models - ERROR - PSF weights file not found. Check that you've specified the correct weights file in $
Traceback (most recent call last):

The files in this directory are:
(tensorflow-2.9.1+py3.10) [uuu68hq@jean-zay2: config]$  ls /gpfswork/rech/ynx/uch76qv/tests/outputs/wf-outputs/wf-outputs-SFE-05nm-seed1/psf_model/
checkpoint  psf_model_physical_poly_no_sfe_id-11_cycle2.data-00000-of-00001  psf_model_physical_poly_no_sfe_id-11_cycle2.index

These files do not have the same id_name specified in /gpfswork/rech/ynx/uch76qv/tests/outputs/wf-outputs/wf-outputs-SFE-05nm-seed1/config/training_config.yaml.
training:
  # ID name
  id_name: sfe_05nm_seed1

Another thing I noticed about this training_config.yaml file is missing the configuration parameter: use_prior.  You need to set this in order to use the Zernike Prior from these datasets else it will not.  You should cancel all of your runs that were deployed, add use_prior=True to the training_config.yaml, and resubmit the jobs.   I just pushed it now to remote.

Hi @jeipollack, I am having this error:
2024-04-16 16:13:56,641 - wavediff - INFO - #
2024-04-16 16:13:56,642 - wavediff - INFO - # Entering wavediff mainMethod()
2024-04-16 16:13:56,642 - wavediff - INFO - #
2024-04-16 16:13:56,642 - wf_psf.utils.read_config - INFO - Loading.../gpfswork/rech/ynx/uch76qv/tests/config/config_SFE_01_seed1/training_config.yaml
2024-04-16 16:13:56,664 - wf_psf.utils.read_config - INFO - Loading.../gpfswork/rech/ynx/uch76qv/tests/config/config_SFE_01_seed1/data_config.yaml
2024-04-16 16:14:00,347 - wavediff - INFO - <wf_psf.utils.configs_handler.TrainingConfigHandler object at 0x14ff0486b370>
2024-04-16 16:14:00,356 - wf_psf.data.training_preprocessing - INFO - Reading in Zernike prior...
2024-04-16 16:14:00,356 - wavediff - ERROR - Check your config file /gpfswork/rech/ynx/uch76qv/tests/config/config_SFE_01_seed1/configs.yaml for errors. Error Msg: 'zernike_prior'.
Traceback (most recent call last):
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/run.py", line 92, in mainMethod
    config_class.run()
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/utils/configs_handler.py", line 202, in run
    train.train(
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/training/train.py", line 295, in train
    psf_model = psf_models.get_psf_model(
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/psf_models/psf_models.py", line 155, in get_psf_model
    return psf_factory_class().get_model_instance(*psf_model_params)
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/psf_models/psf_model_physical_polychromatic.py", line 52, in get_model_instance
    return TFPhysicalPolychromaticField(
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/psf_models/psf_model_physical_polychromatic.py", line 106, in __init__
    self._initialize_parameters_and_layers(
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/psf_models/psf_model_physical_polychromatic.py", line 137, in _initialize_parameters_and_layers
    self._initialize_zernike_parameters(model_params, data)
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/psf_models/psf_model_physical_polychromatic.py", line 155, in _initialize_zernike_parameters
    self.zks_prior = get_zernike_prior(model_params, data)
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/data/training_preprocessing.py", line 342, in get_zernike_prior
    zernike_contribution_list.append(get_np_zk_prior(data))
  File "/gpfswork/rech/ynx/uch76qv/.local/lib/python3.10/site-packages/wf_psf/data/training_preprocessing.py", line 222, in get_np_zk_prior
    data.training_data.dataset["zernike_prior"],
KeyError: 'zernike_prior'
2024-04-16 16:14:00,429 - wavediff - INFO - #
2024-04-16 16:14:00,430 - wavediff - INFO - # Exiting wavediff mainMethod()
2024-04-16 16:14:00,430 - wavediff - INFO - #

Can you check whether your training dataset that you are loading contains the key zernike_prior?

The error I'm encountering is because when I pulled the latest changes you made, I didn't update the dataset in the data_config file.
Is there a way to determine the name of the output of WaveDiff before the execution so the plotting configuration runs automatically after the metrics?

Plotting can run after metrics evaluation for certain set ups.
On Tue, 16 Apr 2024 at 18:03, nadamoukaddem @.***>
wrote:

The error I'm encountering is because when I pulled the latest changes you
made, I didn't update the dataset in the data_config file.
Is there a way to determine the name of the output of WaveDiff before the
execution so the plotting configuration runs automatically after the
metrics?
—
Reply to this email directly, view it on GitHub
https://github.com/CosmoStat/wf-psf/issues/133#issuecomment-2059440003,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXRVNRJSJ3N53CQ5HRRCP3Y5VDTJAVCNFSM6AAAAABFWFMZ2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJZGQ2DAMBQGM
.
You are receiving this because you were mentioned.Message ID:
@.***>


[KYUUBI #1960] Skip creating sparksession and starting engine if the max initialization time exceeds

Why are the changes needed?

close #1960
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[X] Add screenshots for manual tests if appropriate

Test steps:

Start another application to fill the queue.
Modify the stop-application.sh so it doesn't work.
Submit a Kyuubi request.
After waiting for timeout, stop the application in step 1.

We can get the following results:

The kyuubi request exits after timeout.
After obtaining resources, the orphaned engine app fails to execute and exits immediately.

[ ] Run test locally before make a pull request

- Scheduler pool *** FAILED ***
  The code passed to eventually never returned normally. Attempted 660 times over 10.008045327 seconds. Last failure message: 0 was not greater than or equal to 1000. (SchedulerPoolSuite.scala:99)

The failed test doesn't seem to be caused by this pr.

SparkHudiOperationSuite:
2022-03-01 11:42:18,195 ScalaTest-main-running-DiscoverySuite ERROR Filters contains invalid attributes "onMatch", "onMismatch"
2022-03-01 11:42:18,199 ScalaTest-main-running-DiscoverySuite ERROR Filters contains invalid attributes "onMatch", "onMismatch"
- get catalogs
- get schemas
2022-03-01 11:42:28,757 SparkSQLSessionManager-exec-pool: Thread-1794 ERROR Filters contains invalid attributes "onMatch", "onMismatch"
2022-03-01 11:42:28,770 SparkSQLSessionManager-exec-pool: Thread-1794 ERROR Filters contains invalid attributes "onMatch", "onMismatch"
- get tables *** FAILED ***
  rs1.next() was false (HudiMetadataTests.scala:85)
2022-03-01 11:42:53,888 SparkSQLSessionManager-exec-pool: Thread-1857 ERROR Filters contains invalid attributes "onMatch", "onMismatch"
2022-03-01 11:42:53,892 SparkSQLSessionManager-exec-pool: Thread-1857 ERROR Filters contains invalid attributes "onMatch", "onMismatch"
- get columns type *** FAILED ***
  0 did not equal 15 all columns should have been verified (HudiMetadataTests.scala:186)

The failed test doesn't seem to be caused by this pr.

thanks, merging to master


Doesn't work setOnClickListener

Can't set onClick or onLongClick, it doesn't work. But setOnTouchListener works good. What can I do?
Thank you for your good work.

This because TextView is built on top of ScrollView which consumes such events I believe
Sent from my iPhone

On Jan 29, 2016, at 3:50 AM, AseevEIDev<EMAIL_ADDRESS>wrote:
Can't set onClick or onLongClick, it doesn't work But setOnTouchListener works good What can I do?
Thank you for your good work
—
Reply to this email directly or view it on GitHub.

I think so too. I made this methods manually with onTouch. You can make it manually in your class too

Is there anyway to possible fix this? I am having the same issue, my TextViews are in the ScrollView, but its not working.


Pointclouds get dropped

Thank you for your great work with this package, it looks really promising!
I recorded a 3d Pointcloud map of our environment beforhand. And now I'd like to lokalize our robot in said map.
Therefore I wanted to use the ynamic_robot_localization_system_6dof.launch launchfile. But when starting it, after the Map is loaded and global pose estimate is published, all the new Pointclouds get dropped, because:
[ WARN] [1660297010.079226327]: Dropping pointcloud because tf between base_link and odom isn't available
Failed to find match for field 'rgb'.
Failed to find match for field 'normal_x'.
Failed to find match for field 'normal_y'.
Failed to find match for field 'normal_z'.
Failed to find match for field 'curvature'.
Is that a Problem because of the pointcloud type I use? I am using a robosense Lidar and can decide between XYZIRT and XYZI type. So there is no option for rgb... . Am I still able to use this package somehow?

Hello,
If you do not have odometry, then you need to set the odom_frame_id to be the frame as your base_link_frame_id.
Have a nice day :)

Hello,
That error seems to be from sac_model_registration.h#L248, in which the covariance matrix is computed with centroid.hpp#L508.
That class is used in drl in cloud_matcher.hpp#L65 if the max_number_of_ransac_iterations > 0, which is the default for recovery_3d.yaml#L12 (you can set it to 0 to disable the usage of ransac within ICP) and also in sample_consensus_prerejective.hpp#L542
drl removes NaN points from the point clouds when it receives sensor data (localization.hpp#L1953) and after normal estimation (localization.hpp#L2875).
As such, it is odd that the pcl function computeMeanAndCovarianceMatrix is generating a covariance matrix with NaN values.
Keep in mind that the dynamic_robot_localization_system_6dof.launch does not have the initial pose estimation module loaded by default and as such, it expects the end user to specify the initial pose either in that launch file dynamic_robot_localization_system_6dof.launch#L23 or given in a topic dynamic_robot_localization_system.launch#L4
Alternatively, you can enable the initial pose module by commenting the line with the empty.yaml (dynamic_robot_localization_system_6dof.launch#L44) and uncommenting the line below (dynamic_robot_localization_system_6dof.launch#L45)
The 3d reference point cloud can be provided in reference_pointcloud_filename or in reference_pointcloud_topic.
I tested the 6 DoF tracking with the ETHZ ASL dataset using their high complexity environments rosbags. One of the tests can be started with ethzasl_kinect_dataset_high_complexity_slow_fly_movement.launch which should have results similar to this video
If your sensor data has a scale that is very different (objects much smaller or larger than the ones present in the video above), you may need to adjust the search radius / k / correspondence distance values of the files: initial_pose_estimation_3d.yaml, cluttered_environments_dynamic_large_map_3d.yaml and recovery_3d.yaml
Have a nice day :)


Bump version to 2.0.3

What's this PR do?
It bumps version to 2.0.3.

I keep forgetting to include this when creating a PR in Blink. It's slightly different from how we do this in Gambit.


Clarify merge_into documentation

Just a little extra naming in the documentation for merge_into. I hit this twice and because of the way LSP shows the type I always end up digging into the source code to know what I need to add. This is what it currently looks like for me:

Codecov Report

Merging #2153 (365f093) into main (2f37f24) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2153   +/-   ##
=======================================
  Coverage   68.23%   68.23%
=======================================
  Files         134      134
  Lines       16095    16095
=======================================
  Hits        10983    10983
  Misses       5112     5112

Impacted Files
Coverage Δ

src/irmin-fs/unix/irmin_fs_unix.ml
67.74% <0.00%> (-0.65%)
:arrow_down:

src/irmin-test/store.ml
94.95% <0.00%> (+0.05%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Multiple namespace support

Figure out how to get certs for ingresses in different namespaces.

I think I found the solution in this article: https://blog.donbowman.ca/2018/09/06/accessing-a-service-in-a-different-namespace-from-a-single-ingress-in-kubernetes/
KCert can create a service as follows:
kind: Service
apiVersion: v1
metadata:
  name: kcert
  namespace: some-namespace
spec:
  type: ExternalName
  externalName: kcert.default.svc.cluster.local
  ports:
  - port: 80

This will create a service in namespace some-namespace that points to the service in default


opt: improve the performance of sending data

When data reading occurs, there is a high probability of data writeback generated in the OnTraffic.
At this time, it is not necessary to wait for the registration write event callback to immediately perform the write operation, which can improve the performance of sending data.
benchmark use https://github.com/gnet-io/gnet-benchmarks
BEFORE
Using interface lo to connect to [<IP_ADDRESS>]:7002
Ramped up to 500 connections.
Total data sent:     35112.2 MiB<PHONE_NUMBER>2 bytes)
Total data received: 14540.8 MiB<PHONE_NUMBER>3 bytes)
Bandwidth per channel: 13883.816⇅ Mbps (1735477.0 kBps)
Aggregate bandwidth: 4065.857↓, 9817.959↑ Mbps
Packet rate estimate: 372404.8↓, 858515.8↑ (12↓, 41↑ TCP MSS/op)
Test duration: 30.0004 s.

AFTER
Using interface lo to connect to [<IP_ADDRESS>]:7002
Ramped up to 500 connections.
Total data sent:     41217.8 MiB<PHONE_NUMBER>4 bytes)
Total data received: 27600.4 MiB<PHONE_NUMBER>1 bytes)
Bandwidth per channel: 19242.419⇅ Mbps (2405302.3 kBps)
Aggregate bandwidth: 7717.413↓, 11525.006↑ Mbps
Packet rate estimate: 706735.2↓, 999879.1↑ (12↓, 38↑ TCP MSS/op)
Test duration: 30.0009 s.

RESULT
before:
Aggregate bandwidth: 4065.857↓, 9817.959↑ Mbps
after:
Aggregate bandwidth: 7717.413↓, 11525.006↑ Mbps

On what platform did you run the benchmark? Also which version of gnet you were using for benchmark?

 OS: Ubuntu 20.04.6 LTS on Windows 10 x86_64
 Kernel: <IP_ADDRESS>-microsoft-standard-WSL2
 Uptime: 1 day, 12 hours, 8 mins
 Packages: 846 (dpkg), 4 (snap)
 Shell: bash 5.0.17
 Theme: Adwaita [GTK3]
 Icons: Adwaita [GTK3]
 Terminal: Relay(1325)
 CPU: AMD Ryzen 9 7945HX with Radeon Graphics (32) @ 2.495GHz
 GPU: 2060:00:00.0 Microsoft Corporation Device 008e
 Memory: 498MiB / 31951MiB

benchmark version use v2.5.2
This optimization may not be as good as you think. If there is data to send immediately after OnTraffic processing, it will usually go directly into the socket buffer and not into the outbound buffer. Unless the socket is not writable, this change may introduce invalid write operations and cause more CPU consumption.
Perhaps this change will improve intranet services, such as the rpc framework serving intranets.

Write operations are processed first during epoll-wait. If outbound buffer is still not cleared, it means that socket has entered an unwritable state. At this time, after reading operations are processed, socket buffer may have free space for sending, but usually the parsed requests will be placed in new goroutines for processing. If the kernel cannot handle sending in time during this period, an invalid write system call will be wasted. Please close this pr at will. It seems that the upgrade will not be too obvious, but will lead to an increase in cpu overhead.

@panjf2000 这个pr没有从根源上修复问题，只是从表面上优化了发送效率（读一次之后如果有可写就强制发送一次），真正的根源是epoll事件机制出现了问题了
modWrite 通常会包含 read | write 实际上这样干会有两个问题：

一段时间后（约10s之后）read事件触发次数比write触发次数多的多（读多写少）
如果恶意客户端只发不收将会导致服务进程oom（实际上不需要恶意客户端，tcpkali就能稳定复现oom问题）

gnet echo 测试可以稳定复现oom （服务总内存128G，gnet压测30s左右内存峰值110G后几秒被oom）：

[1615784.904275] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=init.scope,mems_allowed=0-1,global_oom,task_memcg=/user.slice/user-1000.slice/session-783.scope,task=echo-gnet-serve,pid=1477890,uid=1000
[1615784.904335] Out of memory: Killed process 1477890 (echo-gnet-serve) total-vm:122038268kB, anon-rss:118361296kB, file-rss:2304kB, shmem-rss:0kB, UID:1000 pgtables:232148kB oom_score_adj:0
[1615789.840989] oom_reaper: reaped process 1477890 (echo-gnet-serve), now anon-rss:0kB, file-rss:2304kB, shmem-rss:0kB

关于这个问题如果要修复应该就是 modWrite时去掉 read事件注册，只保留 write | error 事件，同理 modRead时只保留 read | error 注册
另外：evio 没有这个问题的原因时它不按事件类型进行处理，而是按照自己的状态决定（读优先级最低）：
https://github.com/tidwall/evio/blob/master/evio_unix.go#L229-L246
//fmt.Println("-- loop started --", l.idx)
l.poll.Wait(func(fd int, note interface{}) error {
  if fd == 0 {
	  return loopNote(s, l, note)
  }
  c := l.fdconns[fd]
  switch {
  case c == nil:
	  return loopAccept(s, l, fd)
  case !c.opened:
	  return loopOpened(s, l, c)
  case len(c.out) > 0:
	  return loopWrite(s, l, c) // 如果有可写优先处理可写
  case c.action != None:
	  return loopAction(s, l, c)
  default:
	  return loopRead(s, l, c) // 读取操作优先级最低
  }
})

基于 I/O 多路复用的网络库的模式大多都是基于系统给的事件类型来做相应处理的。至于你说的这种发送快接收慢的客户端，只要是像 gnet 这样把业务层待写的数据保存起来，那就肯定都会有这个问题，即便是不替业务层保存待写的数据，那业务层就要自己保存这些数据，一样会有这个问题，这个也算是这一类网络框架经典的写出数据堆积问题了，性价比最高的一种解决办法是服务端设置一个阀值，业务层每次回写数据的时候都检查一下当前的待写数据堆积是否已经触达这个阀值，为了避免 OOM 通常需要服务端丢弃数据，netty 就是使用的这种方式，它提供了一个 setWriteBufferHighWaterMark 方法。至于 gnet 实现起来就更简单了：目前已经有了 OutboundBuffered() 可以随时检查当前的写出队列的堆积情况了，业务层自己就可以判断了。
至于 evio 的 I/O 处理，反而是非主流的做法，虽然确实能解决这个问题，但因为这种做法每次只处理一种单一的事件，无法充分利用 epoll 的事件通知机制，性能上肯定有所损耗。还有你提出的解决的方法，虽然理论上也能解决这个问题，但是成本不小，一旦那么做了，后续还需要再重新把可读事件加回去，而这需要通过一个系统调用完成，在 LT 模式下待写队列大概率经常会有一些堆积，但是通常能很快被客户端读走，所以绝大部分时候不会出现 OOM。如果使用你说的方式，则需要频繁地删除然后重新添加可读事件，性能肯定下降，这相当于是将某一个极端场景的修复成本强加到大部分正常场景上，这是很不划算的。
还有，对于现实世界来说，性能问题的解决往往不能都丢给服务端，这样只会让服务端的实现越来越复杂，为了覆盖每一个业务问题而导致代码越来膨胀，这通常也是一个不好的实践。拿这个客户端发送快读取慢的问题来说，除了服务端可以使用上述的方法缓解之外，更重要的是对客户端进行改造，平衡好生产和消费的速率，毕竟一个疯狂发送数据而不接收处理数据的客户端有什么实际价值呢？在我看来这种客户端只有一种功能：做性能压测探一探服务端的极限边界，除此之外并没有现实的业务价值。而为了一个没有实际应用价值的极端场景去做一个会降低性能的妥协方案，则更没有价值了。

🤖 Non-English text detected, translating...

Most of the modes of network libraries based on I/O multiplexing are based on the event types given by the system for corresponding processing. As for the client you mentioned that sends quickly and receives slowly, as long as it saves the data to be written in the business layer like gnet, it will definitely have this problem, even if it does not save the data to be written for the business layer. , then the business layer will have to save the data itself, which will still have this problem. This is also a classic problem of data accumulation in this type of network framework. The most cost-effective solution is to set a threshold on the server side, and the business Every time the layer writes back data, it checks whether the current accumulation of data to be written has reached this threshold. In order to avoid OOM, the server usually needs to discard the data. netty uses this method. It provides a setWriteBufferHighWaterMark  method. As for gnet, it is even simpler to implement: there is already OutboundBuffered() that can check the current accumulation of the write queue at any time, and the business layer can judge it by itself.
As for evio's I/O processing, it is a non-mainstream approach. Although it can indeed solve this problem, because this approach only handles a single event at a time, it cannot make full use of the event notification mechanism of epoll. In terms of performance, There must be some loss. There is also the solution you proposed. Although it can theoretically solve this problem, the cost is not small. Once it is done, the readable events need to be added back later, and this needs to be completed through a system call. In LT mode, there is a high probability that there will be some accumulation in the queue to be written, but it can usually be read by the client quickly, so OOM will not occur most of the time. If you use the method you mentioned, you need to frequently delete and re-add readable events, and the performance will definitely decrease. This is equivalent to imposing the repair cost of an extreme scenario on most normal scenarios, which is not cost-effective.
Also, in the real world, the solution to performance problems often cannot be left to the server. This will only make the implementation of the server more and more complex. In order to cover every business problem, the code will become more and more bloated. This is usually also a problem. Bad practice. Take the client's problem of fast sending and slow reading as an example. In addition to the above methods that the server can use to alleviate the problem, it is more important to transform the client to balance the production and consumption rates. After all, a person who sends data crazily cannot What real value is there in a client that doesn't receive and process data? In my opinion, this kind of client has only one function: to do performance stress testing to explore the limits of the server. Other than that, it has no real business value. It is even less valuable to make a compromise that will reduce performance for an extreme scenario that has no practical application value.

这种方式确实有损性能，毕竟不能同时处理读/写了，但是能充分利用tcp的滑动窗口限制对端的发送能力
按照当前的实现，echo 测试已经没办法进行了，基本上都会oom，如果加上丢弃或者断开连接那样也就脱离了echo测试的场景
就看如何取舍了，性能 / 安全