Support for external DNS and NTP servers

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Hi
I'm working with terraform /python scripts to deploy an full ISE environment but I have a private DNS and Domain controller server so I would like to not use Route53 but instead rely on my Active Directory. I have some questions for this:
1- if I comment / unuse the whole Route53 module is the deployment expected to work? Actually terraform deploy ended successfully and state machine function as well but PSN and Secondary Pan seems not registered to primary PAN.
2- I need to set 2 DNS server but the script foresees just to set a single primarynameserver; how can I set a second one?
3- same question for bullet 2, but regarding ntp server.
thanks
Alessio
New or Affected Resource(s)/Data Source(s)
NEW
Potential ISE Automation Configuration
could something like the followings in userdata.tftpl  work?

primarynameserver=${primarynameserver}
secondarynameserver=${secondarynameserver}
primaryntpserver=${primaryntpserver}
secondaryntpserver=${secondaryntpserver}

Hi @bando483 -  Thanks for reaching out, we are checking this and will get back to you

Hi Alessio (@bando483),
To debug this further could you please confirm below points -
1- Is the state machine execution successful?
2- If you have used your private DNS, could you please confirm if the same got configured during ISE installation? To check this you can execute show running-config in all ISE nodes and look for “ip name-server” and “ntp server“.
3- Please confirm is DNS is reachable from all ISE nodes and DNS has correct hosts entries as per installed ISE nodes with correct IPs and deployed ISE nodes are able to resolve other ISE nodes FQDN?
4- Could you please also share Primary Admin node Deployment page UI screen capture?
Please note when we use terraform script to deployment ISE cluster with New VPC, ISE nodes are deployed in Private subnet, where configuring private DNS reachability before deployment starts is not possible until we monitor terraform execution and update VPC/Private subnet to reach to private DNS, so even if terraform execution succeeds, deployment formation will fail.
If we use these script to deploy ISE cluster in existing VPC then Deployment will be successful. (Provided right DNS/NTP are configured in ISE nodes, refer point#2)
As of now till Cisco ISE release 3.3, multiple DNS/NTP in cloud is not supported, We have this feature coming up in Cisco ISE release 3.4 and accordingly terraform scripts and README will be updated to use the same.
Thanks & Regards,
Sudhanshu

Hi
thank for the feedback on multiple DNS/NTP.. i'm gonna wait for next release.
regarding the error, I confirm reachability of DNS is present but I noticed an issue on DNS entries, so I guess this is why the nodes have not been registered. could it be?
by the wait, the state machine ended successfully even though the nodes are not registered and I guess this is a bug since I expected a failure of STate Machine
regards
Alessio

Yes, you are right. If DNS entries are not correct as per ISE deployed ISE nodes, then ISE Deployment node registration will fail.
however, if state machine executed shows success even after failed node registration, then it could be a possible bug. To confirm that could you please share state machine execution logs?
Thanks & Regards,
Sudhanshu

Hi
Sorry, but I have already destroyed the not working deployment.
Do you have any chance to test it on your side?
Regards
Alessio
From: Sudhanshu Sharma @.>
Sent: jeudi 25 juillet 2024 10:58
To: CiscoISE/ciscoise-terraform-automation-aws-nodes @.>
Cc: Bandini Alessio @.>; Mention @.>
Subject: Re: [CiscoISE/ciscoise-terraform-automation-aws-nodes] Support for external DNS and NTP servers (Issue #12)
CAUTION! This message was sent from OUTSIDE of the company.
Please do not provide any confidential information or click on any link and attachment unless you recognize the sender.
Yes, you are right. If DNS entries are not correct as per ISE deployed ISE nodes, then ISE Deployment node registration will fail.
however, if state machine executed shows success even after failed node registration, then it could be a possible bug. To confirm that could you please share state machine execution logs?
Thanks & Regards,
Sudhanshu
—
Reply to this email directly, view it on GitHubhttps://github.com/CiscoISE/ciscoise-terraform-automation-aws-nodes/issues/12#issuecomment-2249823063, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASRQW2OSRV6MY7JZLXOZ2QLZOC4Y7AVCNFSM6AAAAABLDB3K3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBZHAZDGMBWGM.
You are receiving this because you were mentioned.Message ID: @.***>

Hi
I deployed an infrastructure on Milan Region with 2 PAN/MTRs and 4 PSNs.
the deploy ended successfully via terraform and all the nodes can reach the DNS and NTP and can be pinged
The State Machine ended with this error; please see the screenshots below
@.***
@.***
Can you please support me?
Alessio
From: Sudhanshu Sharma @.>
Sent: lundi 22 juillet 2024 20:39
To: CiscoISE/ciscoise-terraform-automation-aws-nodes @.>
Cc: Bandini Alessio @.>; Mention @.>
Subject: Re: [CiscoISE/ciscoise-terraform-automation-aws-nodes] Support for external DNS and NTP servers (Issue #12)
CAUTION! This message was sent from OUTSIDE of the company.
Please do not provide any confidential information or click on any link and attachment unless you recognize the sender.
Hi Alessio @.***https://github.com/bando483),
To debug this further could you please confirm below points -
1- Is the state machine execution successful?
2- If you have used your private DNS, could you please confirm if the same got configured during ISE installation? To check this you can execute show running-config in all ISE nodes and look for “ip name-server” and “ntp server“.
3- Please confirm is DNS is reachable from all ISE nodes and DNS has correct hosts entries as per installed ISE nodes with correct IPs and deployed ISE nodes are able to resolve other ISE nodes FQDN?
4- Could you please also share Primary Admin node Deployment page UI screen capture?
Please note when we use terraform script to deployment ISE cluster with New VPC, ISE nodes are deployed in Private subnet, where configuring private DNS reachability before deployment starts is not possible until we monitor terraform execution and update VPC/Private subnet to reach to private DNS, so even if terraform execution succeeds, deployment formation will fail.
If we use these script to deploy ISE cluster in existing VPC then Deployment will be successful. (Provided right DNS/NTP are configured in ISE nodes, refer point#2)
As of now till Cisco ISE release 3.3, multiple DNS/NTP in cloud is not supported, We have this feature coming up in Cisco ISE release 3.4 and accordingly terraform scripts and README will be updated to use the same.
Thanks & Regards,
Sudhanshu
—
Reply to this email directly, view it on GitHubhttps://github.com/CiscoISE/ciscoise-terraform-automation-aws-nodes/issues/12#issuecomment-2243583212, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASRQW2IT7UF4IX4MSQ5MOKDZNVGS3AVCNFSM6AAAAABLDB3K3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGU4DGMRRGI.
You are receiving this because you were mentioned.Message ID: @.***>

I do not see screen capture here Alessio(@bando483 ), Could you please help me with the cloudwatch logs for failed state to debug the issue.

Update - I am able to reproduce the issue where Step Function work flow succeeds even if node registration fails. This is a bug in code , we will fix it by enhancing error Handling for register node state.

Have you already fixed the bug and the code is already available? I would like to use more stable code in my production deploy.. expected for tomorrow end of day Italy time
Thanks
Inviato da iPhone
Il giorno 28 lug 2024, alle ore 10:42, Sudhanshu Sharma @.***> ha scritto:

CAUTION! This message was sent from OUTSIDE of the company.
Please do not provide any confidential information or click on any link and attachment unless you recognize the sender.
Update - I am able to reproduce the issue where Step Function work flow succeeds even if node registration fails. This is a bug in code , we will fix it by enhancing error Handling for register node state.
—
Reply to this email directly, view it on GitHubhttps://github.com/CiscoISE/ciscoise-terraform-automation-aws-nodes/issues/12#issuecomment-2254396896, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASRQW2LDHD4BN5LVP6RF74TZOSVGVAVCNFSM6AAAAABLDB3K3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJUGM4TMOBZGY.
You are receiving this because you were mentioned.Message ID: @.***>

Fix is in progress, we will update once changes are merged.


/go/pkg/mod/github.com/rsocket/rsocket-go@v0.5.13/rx/subscriber.go:12:60: cannot use func literal (type func(interface {})) as type func(interface {}) error in argument to reactor.OnNext

/go/pkg/mod/github.com/rsocket/rsocket-go@v0.5.13/rx/subscriber.go:12:60: cannot use func literal (type func(interface {})) as type func(interface {}) error in argument to reactor.OnNext
go.mod
module rmw

go 1.15

require (
        github.com/golang/protobuf v1.4.2
        github.com/jjeffcaii/reactor-go v0.2.2
        github.com/rsocket/rsocket-go v0.5.13
        github.com/rsocket/rsocket-rpc-go v0.0.1
        github.com/stretchr/testify v1.6.1
)

Thanks for feedback, rsocket-go v0.5.x should use github.com/jjeffcaii/reactor-go v0.1.x.
I'll release v0.6.x and it will bump reactor-go 0.2.x In the near future.
Please try to switch github.com/jjeffcaii/reactor-go v0.2.2 to v0.1.4
module rmw

go 1.15

require (
        github.com/golang/protobuf v1.4.2
        github.com/jjeffcaii/reactor-go v0.1.4
        github.com/rsocket/rsocket-go v0.5.13
        github.com/rsocket/rsocket-rpc-go v0.0.1
        github.com/stretchr/testify v1.6.1
)

thanks


RestKit 0.25 xcode 7 Version 7.0 (7A220) compiler warnings

Method override for the designated initializer of the superclass '-init' not found
On all subclass

This is a duplicate issue. Please file a PR with a fix?


Rename group_rows_by to group_rows

This is more consistent with the functions for sorting and filtering. It's also useful for #612.

:tada: This issue has been resolved in version 0.22.0 :tada:
The release is available on:

v0.22.0
GitHub release

Your semantic-release bot :package::rocket:


FFprobeTimeout on very simple configurations, but container internal ffmpeg command works.

Hello, I'm having a hard time connecting to the rtsp cameras.
Here is my configuration:
## Start by adding some cameras
ffmpeg:
  camera:
    camera_1:  # This value has to be unique across all cameras
      name: Camera 1
      host: NVR-P.lan
      port: 554
      path: '/cam/realmonitor?channel=1&subtype=0'
      stream_format: rtsp
      protocol: rtsp
      rtsp_transport: tcp
      username: REDACTED
      password: REDACTED

darknet:
  object_detector:
    cameras:
      camera_1:  # Attach detector to the configured camera_1 above
        fps: 1
        scan_on_motion_only: false  # Scan for objects even when there is no motion
        labels:
          - label: person
            confidence: 0.75
            trigger_recorder: true

nvr:
  camera_1:  # Run NVR for camera_1

I get these errors:
[2024-05-16 08:40:13] [ERROR   ] [viseron.components.ffmpeg.stream.camera_1] - Retrying None in 16.0 seconds as it raised FFprobeTimeout: FFprobe command timed out after 60s.

When I get into the shell of the container, this command outputs encoded stuff:
ffmpeg -hide_banner -loglevel error -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -use_wallclock_as_timestamps 1 -vsync 0 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -rtsp_transport tcp -i 'rtsp://REDACTED:REDACTED@NVR-P.lan:554/cam/realmonitor?channel=1&subtype=0' -f rawvideo -pix_fmt nv12 pipe:1

I have also tried using the raw_command option but no luck.
I have also tried version 2.2.0, 2.3.0, and 3.0.0b4, and they all have this error.
So, there is no connectivity issues per se. Am I missing something in the configuration?

Never mind, I got it to work by adding this:
      fps: 6
      width: 1920
      height: 1080
      codec: h264
      audio_codec: null

Yeah something is definitely off with ffprobe.
Have not been able to figure out what. I should move to something else.
Glad you got it sorted


Promise rejection inside HubConnection.js when closing "early"

I am using alpha 2 at the moment and there is a situation in my app where shortly after being connected I close the connection. If this happens, I get an unhandled rejection of a promise with the errror Error: Invocation cancelled due to connection being closed. The stack trace is:
    at callbacks.set (HubConnection.js:157)
    at callbacks.forEach.callback (HubConnection.js:93)
    at Map.forEach (<anonymous>)
    at HubConnection.connectionClosed (HubConnection.js:92)
    at HttpConnection.HubConnection.connection.onclose (HubConnection.js:43)
    at HttpConnection.stopConnection (HttpConnection.js:122)
    at HttpConnection.eval (HttpConnection.js:112)
    at Generator.next (<anonymous>)
    at fulfilled (HttpConnection.js:6)
    at <anonymous>

When a hub connection is closed either by the server or by the user all pending invocations are terminated with the error you posted. The promise cannot be fulfilled without the result and there will never be a result because the connection is closed, so it is rejected. What is your expectation here?

My expectation is that either the rejection is catched or I as a consumer of the library get the possibility to catch it. Throwing an exception which cannot be caught should not be the behaviour when normal operations occur.

When you invoke a hub method the Promise is returned to you. You can then use .catch on this promise to handle the exception.

Ah, you mean invoke of signalR.HubConnection returns that promise which is rejected here? I did not recognize that and it are those promises which are rejected here.
So I am now totally able to catch the situation, thank you!
Hoping for docs and typings soon :)

This is correct. You also use this promise to get the result of non-void hub methods.


Adding breakpoint feature for django-html and django-txt template files which belong to django extension

as django debugging is present in html, jinja i request to add support for addition launguages like django-html and django-txt which belong django vscode extension which can help in debugging .djt file

I already made a pull request #19288 which can enable break point for these languages mentioned above without any changes to the code base

resolved in #19288 thank you ✔😊


Support of 4-digit hex color format in helpers

Hello everyone!
I'm using many of the style helpers that offers this library in a CSS-IN-JSS context. The thing is that I have a color picker that generates color strings and the output is used for the polished helpers. Sometimes the user produces 4-digit in hex format. But the helpers of this library don't support that, or at least darken which is the one I tested. It throws the next error:
Couldn't parse the color string. Please provide the color as a string in hex, rgb, rgba, hsl or hsla notation.

polished: 2.0.3
react: 16.4.2
JSS-in_CSS: none, just using a helper.

Reproduction
I prepared a reproduction to show it, just press the 4-digit hex color and you will get the error displayed.
https://codesandbox.io/embed/qv3opq0wv6?view=preview

Thanks for reporting this @Gywem. All the color helpers use the same module to process color values and we did not add 4 digit shorthand support when we added support for 8-Digit, but this should be relatively trivial to implement in a future release.


[move source language] Added Borrow Checker Tests. Fixed Self Cycles in Borrow Graphs

Motivation

Added tests for borrow checker in the source language
Fixed an issue with both borrow graph implementations around
cycles that could create self cycles.

Added tests to document these cases

Test Plan
wrote em

@bors-libra r=tzakian

:pushpin: Commit ffb939c has been approved by tzakian

:hourglass: Testing commit ffb939c35d790987aacbefb30b218ecda6155524 with merge fd139d3492cf9896337b61f29b65b32682fc8c67...

:sunny: Test successful - checks-circle_commit_workflow
Approved by: tzakian
Pushing fd139d3492cf9896337b61f29b65b32682fc8c67 to master...


fzf-tmux -p not working on tmux 3.3a-3

[x] I have read through the manual page (man fzf)
[x] I have the latest version of fzf
[x] I have searched through the existing issues

Info

OS

[x] Linux
[ ] Mac OS X
[ ] Windows
[ ] Etc.

Shell

[ ] bash
[x] zsh
[ ] fish

Problem / Steps to reproduce
After upgrading to fzf 0.39.0, the command fzf-tmux -p gives the following error message.
/usr/bin/fzf-tmux: line 183: bc: command not found
fzf-tmux: tmux 3.2 or above is required for popup mode

tmux -V gives tmux 3.3a, so there shouldn't be an issue with my tmux version.

I assumed bc is installed on most systems by default, but it looks like I was wrong.

Please test if f1a9629 fixes your problem.

Thanks for confirmation. Maybe we should try with awk first? What do you think?

Given that awk is a coreutil I think it would be generally safer to assume that people have it than bc. What was the reasoning behind the use of bc? I'm not particularly well-versed on either choice so I wouldn't be able to say which is the better one.

What was the reasoning behind the use of bc?

Because it's usually the go-to choice for arithmetic operations. But what we're doing here is very simple, and awk is capable of doing it. I'll think about it. Thanks.


Possible Bug - Exception when load characters with image loaded by url on Kanka

Environment Details
Foundry VTT Version: 0.8.8
Operating System: [Linux]
How Are You Using Foundry: [Linux,Native Application (Electron), Chrome]
Which Game System: Dungeons and Dragons 5th Edition, version 1.3.6
Modules Enabled?: Yes (if you need i can give you the full list)
Module version: 3.2.1
OS, Hosting, Browser (if applicable): Linux/Node/Chrome
Description of the issue:
So I checked the bug with the 'Characters' category, but it might be present in the others as well.
Kanka provides two methods to upload images either by uploading from a local image or via web url.
In the first case everything works perfectly in the second when I make the 'link' of the character (with the image retrieved from a web url on Kanka) I import it successfully, but without the image, if I click on the 'update' it gives me an exception that I report below.

It's strange because I don't understand at the import level what can change, in both upload cases Kanka should create her own web link for the entity in question.
This is one of the url generated by Kanka that gives the error:
https://images.kanka.io/user/k9UFBC3o61jsoxuMoTCKJDEnn00=/smart/src/characters%2F60fc48a705efd_qoh1hnwpng?webpfallback
as you can see the ".png" extension is "hidden" with the text '' nwpng "which is the cause of the error, the browser then realizes that it is a valid image and displays it correctly if you use the link.

Oh, very interesting! Thanks for bringing this up. I have to investigate how / why kanka creates these URLs without those file endings. I might have to bring this up on the kanka discord or ignore these cases, because it seems foundry expects a valid file ending when setting the image of a journal entry.
I'll have to investigate this further. Thanks again!

I checked this with the Kanka team. This is currently a bug with images imported via URL. They will fix this with the next Kanka release, but only for new images. Images that are already broken (which are missing the proper file extension) won't be fixed. So to be able to properly import them you'll have to add the images again after the next update of their platform.
One "easy" way would be to download the image from kanka and upload it as a new image for the broken entity. You could probably already do this now to fix them, as uploaded images don't seem to be problematic in that regard right now.
I'll close the issue now as there doesn't seem to be anything else I could do on my end to fix this as this right now. Thanks for bringing this to my attention!

Ty for the info , very fast !


bug: CollectionRepeat wrong event 'scroll.resize'

Type: bug
Platform: all
The following code hook the wrong event 'scroll.resize'. The JQuery event name is event+namespace. So this event will be interpreted as 'scroll' event, which will be triggered for every scrolling. Because the scroll view will trigger the scroll event through the "triggerScrollEvent()".
The consequence is the "CollectionRepeat" will always be forced to refresh the layout for each scrolling. This will waste the performance.
// Dimensions are refreshed on resize or data change.
scrollCtrl.$element.on('scroll.resize', refreshDimensions);</span>

Hey @dreamershl, That would be a big problem. Can you put together a codepen demonstrating that refreshDimensions() is indeed firing every scroll event?

Perry
http://wenxuebu.com/forex-dollar
24.03.2015, 19:30, "Perry Govier" notifications@github.com:Hey , That would be a big problem. Can you put together a codepen demonstrating that refreshDimensions() is indeed firing every scroll event?
—Reply to this email directly or .

Hi, because of my unfamiliarity to the codepen, I just paste the screen capture of my current environment for your reference. 1 is the detail call stack analysis, another one is my working environment for your reference.
When scrolling the list, the following is the key call path
self.__publish(scrollLeft, scrollTop, level);
           -> self.triggerScrollEvent();
                       ->refreshDimensions();
                                ->getRepeatManager().refreshLayout();

All because of this event triggering "ionic.trigger('scroll', {" in the  triggerScrollEvent(). It triggers a global scroll event (no namespace as suffix). So the current event handler "scrollCtrl.$element.on('scroll.resize', refreshDimensions)" will be invoked.
If correct the event name from "scroll.resize" to "resize.scroll". The "refreshDimensions" will be only invoked when the element is resized. Just FYI, there are 3 places to be corrected.
Last one, may I request a same small enhancement in this new version again? Please export the current shown items. If it does, the application get chance to dynamically load the data. This is extremely helpful on the real time apps performance such as stock price updates.
To simplify the export,
1: instead of the closure, put the 'render()' as the method of the "RepeatController"
this.render=function(forceRerender) {

2: return the "itemsShownMap" from the "this.render"
Then the application get chance to know the current visible items exactly!
Call stacks:

My working environment:

Good catch. jQuery will clobber our DOM events with . in them, noted.


PrecompileSwiftBridgingHeader build command failure [iOS][0.61]

The build on iOS fails after upgrading to 0.61. The following error is being logged out:
/Users/PATH_TO_THE_PROJECT/ios/Pods/boost-for-react-native/boost/tr1/detail/config_all.hpp:158:17: note: in file included from /Users/PATH_TO_THE_PROJECT/ios/Pods/boost-for-react-native/boost/tr1/detail/config_all.hpp:158:
#  include_next <utility>
                ^
/Users/PATH_TO_THE_PROJECT/ios/Pods/boost-for-react-native/boost/tr1/tr1/utility:21:20: error: 'utility' file not found
#     include_next <utility>
                   ^
2 warnings and 6 errors generated.
<unknown>:0: error: failed to emit precompiled header '/Users/PATH_TO_THE_PROJECT/ios/build/PROJECT_NAME/Build/Intermediates.noindex/PrecompiledHeaders/PROJECT_NAME-Bridging-Header-swift_2W30XBHEK5XUD-clang_2QN4BRVK8G9LI.pch' for bridging header '/Users/PATH_TO_THE_PROJECT/ios/PROJECT_NAME-Bridging-Header.h'

<unknown>:0: error: generate-pch command failed with exit code 1 (use -v to see invocation)

** BUILD FAILED **

The following build commands failed:
	PrecompileSwiftBridgingHeader normal x86_64
	CompileSwiftSources normal x86_64 com.apple.xcode.tools.swift.compiler
(2 failures)

error Command failed with exit code 1.

React Native version:
System:
    OS: macOS Mojave 10.14.4
    CPU: (4) x64 Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz
    Memory: 559.85 MB / 16.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 8.10.0 - ~/.nvm/versions/node/v8.10.0/bin/node
    Yarn: 1.17.3 - /usr/local/bin/yarn
    npm: 5.6.0 - ~/.nvm/versions/node/v8.10.0/bin/npm
    Watchman: 4.7.0 - /usr/local/bin/watchman
  SDKs:
    iOS SDK:
      Platforms: iOS 13.0, DriverKit 19.0, macOS 10.15, tvOS 13.0, watchOS 6.0
    Android SDK:
      API Levels: 26, 27, 28, 29
      Build Tools: 25.0.0, 26.0.2, 27.0.2, 27.0.3, 28.0.3, 29.0.2
      System Images: android-28 | Google APIs Intel x86 Atom, android-29 | Google Play Intel x86 Atom
  IDEs:
    Android Studio: 3.0 AI-171.4443003
    Xcode: 11.0/11A420a - /usr/bin/xcodebuild
  npmPackages:
    @react-native-community/cli: ^1.11.2 => 1.11.2
    react: 16.9.0 => 16.9.0
    react-native: ^0.61.0 => 0.61.0
  npmGlobalPackages:
    react-native-generate: 0.1.0
    react-native-git-upgrade: 0.2.7

Steps To Reproduce

Upgrade RN to 0.61
Run react-native run-ios
The build fails with the error mentioned above

Describe what you expected to happen:
Expecting successful compilation.

Resolved after removing redundant entries in HEADER_SEARCH_PATHS

Hi @badeleux could you please more specific, and point out where those settings are actually located? Thanks!


Does lightgbm_predict4j suit for the model generated by lightgbm v2.0.5 or higher?

Noticed here exists a demo suit for v2, and i can run the UseageTest in v2 package successfully, while when i use a model generared by lightgbm v2.0.5, it produced a error: "[Tree] Tree model string format error, should contain default_value field". The default_value is contained in models generated by lightgbm's earlier version. The model file cause the error is attached below.
Thanks for any kind help in advance!
lgbm_test.txt

https://github.com/lyg5623/lightgbm_predict4j/pull/4
yeah, thanks a lot for your positive contribution, it seems like that the default values are set to zeros as what i tried above. Is that reasonable or explainable from you point?

What you set as default_value, i.e. what the value should take if it's missing, should be up to the modeller really - I haven't looked into the lightgbm implementation in any detail to see how missing values are handled. It seems that default_value is entirely deprecated in the newer lightgbm releases (and therefore it's also not in the model file when saved from lightgbm). lightgbm_predict4j is behind in that it still expects it.
The way, I've included it, is that you can still include default_value in the model if you want to set it. It's set to 0.0 if not set otherwise.
It might be worthwhile to make sure lightgbm_predict4j (particularly v2) reflects the latest version of the lightgbm format.

@benman1 Hello! Why feed different features to the model and the predicted results are the same?

Please explain clearly what you mean, maybe give an example, show your code, etc; otherwise, I am afraid it's not possible to answer.

@benman1 What does default_value mean?
17:46:12.531 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.532 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.533 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.533 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.534 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.535 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.535 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.536 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.536 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.536 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.537 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.537 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.537 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.538 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.539 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.539 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.540 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.541 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
I suspect this is the problem.

I see ;)

In addition, I found that this prediction code has scoring errors when making binary prediction, for example, the label-1 sample probability score will be less than that labele-0 sample.

In addition, I found that this prediction code has scoring errors when making binary prediction, for example, the label-1 sample probability score will be less than that labele-0 sample.
label           features             lightgbm_predict4j_score         lightgbm_spark_score
1         0.0,0.0,1.0,1.0...         0.5108296434776519            0.4907534297184635
0        0.0,0.0,1.0,0.0...         0.5115104195454456             0.48705341451927436

@benman1 Hello! In addition, I found that this prediction code has scoring errors when making binary prediction, for example, the label-1 sample probability score will be less than that labele-0 sample.
label				features					lightgbm_predict4j_score				lightgbm_spark_score
1					0.0,0.0,1.0,1.0... 			0.5108296434776519						0.4907534297184635
0					0.0,0.0,1.0,0.0... 			0.5115104195454456 						0.48705341451927436

@benman1 Hello! In addition, I found that this prediction code has scoring errors when making binary prediction, for example, the label-1 sample probability score will be less than that labele-0 sample.

@benman1 Hello! In addition, I found that this prediction code has scoring errors when making binary prediction, for example, the label-1 sample probability score will be less than that labele-0 sample.

Excuse me, is this problem related to default_value?
Looking forward to your reply~

I asked you for a clearer description of the problem. I still haven't seen
it. I can't answer your question unless you explain clearly. For example,
could you break it down so I can replicate it?
On Fri, 26 Apr 2019, 13:19 周杰<EMAIL_ADDRESS>wrote:

@benman1 https://github.com/benman1 Hello! In addition, I found that
this prediction code has scoring errors when making binary prediction, for
example, the label-1 sample probability score will be less than that
labele-0 sample.
[image: image]
https://user-images.githubusercontent.com/10007145/56806977-13368500-6860-11e9-8885-35825eb3eaad.png
Excuse me, is this problem related to default_value?
Looking forward to your reply~
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lyg5623/lightgbm_predict4j/issues/3#issuecomment-487036898,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACSJO7EAVDARU3DSZCDXAA3PSLXLXANCNFSM4EMKWJOA
.

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

No problem. I am happy you find the code useful.
On Sun, 28 Apr 2019, 03:38 周杰<EMAIL_ADDRESS>wrote:

@benman1 https://github.com/benman1 I'm really sorry for not explaining
my problem clearly. At present, I have found the problem, from my code.
Thank you very much for your recent help. Thank you very much.~
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lyg5623/lightgbm_predict4j/issues/3#issuecomment-487335902,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACSJO7AB7FR4DJHBO6ILZ7TPSUEZVANCNFSM4EMKWJOA
.

兄弟，这个问题有结论了吗？lightgbm_predict4j到底能否适用于lightgbm v2.0.5更高版本的模型？

I was having trouble with lgbm model 2.3.2 (2020 May) and had defaultvalue null array exception.
changing all defaultValue[node] to 0 helped me.
anyways, thank you for the issue raising here!
( LGBM is so huge, and I don't understand why people don't have much interest on developing java version. All of the people are talking about AI stuff and writing a book, but never take themselves into deep level of a production. )

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

hello, do you remember what the problem is in your code. I got the difference between lightgbm_predict4j and lightgbm_python, and i dont know what's wrong with my code.

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

hello, do you remember what the problem is in your code. I got the difference between lightgbm_predict4j and lightgbm_python, and i dont know what's wrong with my code.

i know what happened...this java implement does not support "category feature"..


[BUG] unable to open nano editor

I've just installed the library in my Laravel 8 project via composer, published the configuration, and changed the config.
I ran my project. it works great. but when I try to remove any file, I got permission denied error.
Also, while opening nano editor I got issues like, sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
Screenshots

You get permissions denied issue, because the user which ran the web service (probably apache, httpd, nginx) doesn't have a permission on that part.
About second message: use -S option  and see what happens

I provided 777 permission still got denial issue.
Also, tried nano editor with -S option still got issue (sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper).

@queensherainfotech I think this is more server configs related issue. I didn't get that issue on my local server and VPS.
Can you please contact hosting support?


getURL not wokring for native

I have openfl.Lib.getURL not working on native targets with the latest OpenFL version.
And I think, I know why. :smile:
Any chance it to be fixed?
Thank you!
    public static function getURL (request:URLRequest, target:String = null) {

            if (target == null) {

                target = "_blank";

            }

            #if (js && html5)
            Browser.window.open (request.url, target);
            #elseif flash
            return flash.Lib.getURL (request, target);
            #end

        }

Eah, what about Android, iOS and etc?

I also need this functionality and i've been looking into lime.tools.helpers.ProcessHelper to see how it's done:
	public static function openURL (url:String):Void {

		if (PlatformHelper.hostPlatform == Platform.WINDOWS) {

			runCommand ("", "start", [ url ]);

		} else if (PlatformHelper.hostPlatform == Platform.MAC) {

			runCommand ("", "/usr/bin/open", [ url ]);

		} else {

			runCommand ("", "/usr/bin/xdg-open", [ url, "&" ]);

		}

	}

Is there a specific command that can be used here to open native browsers in iOS and Android? Or do i have to include methods in pure java / obj-c?
Any tips to help direct my efforts will be greatly appreciated! :)

This is supported in current openfl.Lib.getURL or lime.system.System.openURL and lime.system.System.openFile :smile:

Any tips on how to hack support for this in OpenFL 3.6.1 and Lime 2.9.1?
I'm stuck with these versions for a project and really need to fix this!
Thanks!

Desktop or mobile?

Mobile

Are you using legacy? I thought it had this support on iOS/Android

No, using next. I believe the methods are not implemented for iOS and Android (see previous comment from November, 1st above)

Then you'll likely want to look at adapting this commit: 😄
https://github.com/openfl/lime/commit/54a8f32f60a3a6c00d985be17eb98c8ffd101078

Thanks @jgranick !

@jgranick One more question Joshua: i'm using an altered version of the release versions (3.6.1 and 2.9.1) of OpenFL and Lime, but to make the changes i need the dev version of them right? Is there anything else i need to add besides the project folder?
Thanks!

@Tiago-Ling You can try using a "legacy" version of the Lime project directory, and see if it builds for you. I have a feeling it will probably work as a drop-in for your current version, the next step would be making modifications to the source files there to try and include the additional API end points I showed from the newer version.
Current Lime releases have a DRASTICALLY different project/ directory (and resulting C++ binary) and would not be compatible, so you'll have to modify legacy project/ sources instead 😦

Commit you mentioned is indeed "legacy" only, but i'm using "next". Is there a commit that adds support for openURL on mobile using next?
Thanks!


fix(pkg/conf): skip ..data symlink reading creds

Summary of Changes
This fixes the remaining problems described in #379 mentioned by @jackfrancis where the deis-builder pod enters a CrashLoop with the following error:
Error getting storage parameters (read /var/run/secrets/deis/objectstore/creds/..data: is a directory)`

The creds directory structure might look like this (at least on k8s 1.3.0):
drwxrwxrwt. 3 root root  200 Jul  9 14:49 .
drwxr-xr-x. 3 root root 4.0K Jul  9 14:50 ..
drwxr-xr-x. 2 root root  160 Jul  9 14:49 ..7987_09_07_14_49_31.664629063
lrwxrwxrwx. 1 root root   31 Jul  9 14:49 ..data -> ..7987_09_07_14_49_31.664629063
lrwxrwxrwx. 1 root root   16 Jul  9 14:49 accesskey -> ..data/accesskey
lrwxrwxrwx. 1 root root   21 Jul  9 14:49 builder-bucket -> ..data/builder-bucket
lrwxrwxrwx. 1 root root   22 Jul  9 14:49 database-bucket -> ..data/database-bucket
lrwxrwxrwx. 1 root root   13 Jul  9 14:49 region -> ..data/region
lrwxrwxrwx. 1 root root   22 Jul  9 14:49 registry-bucket -> ..data/registry-bucket
lrwxrwxrwx. 1 root root   16 Jul  9 14:49 secretkey -> ..data/secretkey

The problem is that the credential reading code choked on the dir ..7987_09_07_14_49_31.664629063 which was fixed by @bacongobbler in PR #382.
This second fix also skips the ..data symlink, which points to a directory but is a symlink, so IsDir() returns false.
A more generic solution to this fix would be to check for a symlink, use Readlink() and then check for a directory, maybe even recursive. Unless kubernetes adds another level of indirection to the credentials structure, I think that's overkill.
Issue(s) that this PR Closes
Closes #379
Testing Instructions
Please provide a detailed list for how to test the changes in this PR.

Create a Deis Cluster on kubernetes 1.3.0
Edit the deis-builder rc to point to a deis/builder image with this fix
Delete all existing pods belonging to the deis-builder rc
Check if the replacement deis-builder pods come up with status running

There is a fixed build in quay.io/felixbuenemann/builder:git-49085c0 which can be used for quick testing.

Thanks for the contribution! Please ensure your commits follow our style guide. This code will be tested once a Deis maintainer reviews it.

[Current coverage][cc-pull] is 41.40%

Merging [#384][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.17%

@@             master       #384   diff @@
==========================================
  Files            25         25
  Lines          1140       1140
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits            474        472     -2
- Misses          638        640     +2
  Partials         28         28

Powered by Codecov. Last updated by [80d8cf9...08a89d8][cc-compare]
[cc-base-branch]: https://codecov.io/gh/deis/builder/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/deis/builder/compare/80d8cf922b555f263251cd7963f626dabcfc8588...08a89d8824844b05c1e8080021aeb6834b5b77e9?src=pr
[cc-pull]: https://codecov.io/gh/deis/builder/pull/384?src=pr

I tested this with a k8s 1.3.0 GKE cluster on top of builder master and it does fix the issue. Also works with k8s 1.2.5 implicitly because that's still the CI environment.
Thanks @felixbuenemann!


Fix crash in - [ARTiledImageView drawRect:], line 128

Version: 2.4.1 (2016.03.22) | net.artsy.artsy
Stacktrace
ARTiledImageView;drawRect:;ARTiledImageView.m;128
ARTiledImageViewWithBreadcrumb;drawRect:;ARZoomView.m;38
Reason
objc_msgSend() selector name: displayTileBorders
Link to HockeyApp

https://rink.hockeyapp.net/manage/apps/37029/crash_reasons/119997224

Closing in favour of #773.


Window is not focused automatically when it's activated

To Reproduce
Steps to reproduce the behavior:

Open app
Window is opened
Try to navigate with keyboard
Nothing happens

Or

Open app.
Focus any focusable control with mouse.
Minimize window
Try to navigate with keyboard
Nothing happens

Expected behavior
Window should be either focusable by default or focus first focusable (last focused?) control on activation.
Desktop (please complete the following information):

OS: Windows
Version: https://github.com/AvaloniaUI/Avalonia/pull/11287 and newer

@grokys I fixed similar bug for DataGrid, but not sure about best approach with Window yet.


Vickel: Watch northlandz.com

Vickel requests the watch of the watch_keyword northlandz\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

northlandz\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ


Win 10 unable to run binary

I have downloaded the binary in gedcom-windows.zip V39.5.3.
Windows 10 security will not let it run even when I tell it to run it anyway.
Any ideas?

Ok I get it. It is a command line tool. OOOOOOPS!


Instal memcached on windows

Hi,
Is there any way to avoid the build during installation of memcached on windows?
Thx!

No, because the hashring requires 64bit bitshifthing which is not possible in JavaScript and therefor requires a binary add-on so we can correctly distribute the servers in a cluster. We could make it an optional dependency but it means that when it fails to build you can only use one server instead of multiple servers and I don't think that this makes sense.

Why can't the binary be pre-built and distributed with the package?

Because it would have to be pre-build for every single operating system and node.js version combination and bundled within the package. So instead of downloading a 2mb file you start downloading a 200mb file because it's all bundled in. That's not really a solution. There is however a project called node-pre-gyp which makes it possible to create pre-build binaries and they are then stored on s3 but this is hosting you have to pay for your self and quit hard to setup and not really worth the time as I beleive that these are issues that npm needs resolve, not the module authors.

I just use VirtualBox running a linux distro with hassle free memcached server during development on my windows machine. Works wonders

@3rd-Eden first, there has got to be a way to install an architecture or os specific binary through npm rather than downloading 200mb of binaries, no? Second, how would you recommend approaching a server deployment of an application that has a mem-cached dependency?
Thx!

@morious I'm just hoping that npm solves this problem. They should just pre-compile these kind of binary dependencies on their server and use those during installation. But I highly doubt that's something they are willing pursue. The node-gyp-pre module is a good initiative except for the part when you have the host the files your self and it's quite (read super) painful to setup.
Ideally, someone should write a module which just publishes the compiled binaries to npm with the suffix of the build info like memcached-<namespace>-<os fluff> and then magically downloads them using an install script in the package.json.

@3rd-Eden I agree with both your suggestions.
A question regarding my current case, I am using a win 7 dev machine and my target server is Win 2008 server. How would you recommend i approach deploying my application without having to run through the build process? I was thinking to run the build on my local machine and just copy the mem-cached folder to the server. Is that the right approach?

@morious I can't really advice on that as I'm not Windows user.

@3rd-Eden solved it by:

Build on a local dev machine
copy folder to server
point to relative path in require statement (instead of installing with npm)

Thx!

Hello morious.
How did you manage to install the memcached module on Win7?
When I tried to do so, I got an error:
https://github.com/3rd-Eden/memcached/issues/246


Add guide for system dependency for bootstrap

Hi,
I am trying out the example inside a relative new VM (ubuntu16.04).
The cargo test report a linker error at the end.
  = note: /usr/bin/x86_64-linux-gnu-ld: cannot find -lsqlite3
          /usr/bin/x86_64-linux-gnu-ld: cannot find -lpq
          /usr/bin/x86_64-linux-gnu-ld: cannot find -lmysqlclient
          collect2: error: ld returned 1 exit status

Is it possible to update the README to include some steps for a brand new linux VM setup?

Hi!  Feel free to submit a PR for a more detailed README.  I have a link to Diesel for error cases (http://diesel.rs/guides/getting-started/), but having that info in the README would be beneficial.

I fixed this by
sudo apt install libsqlite3-dev

@flyq Thanks.


Trouble with pip install bigbert

OS: Microsoft Windows 10
Stack trace:
(base) C:\Users\nicway\School>pip install bigbert
Collecting bigbert
Using cached bigbert-0.1.0-py3-none-any.whl (7.1 MB)
Collecting symspellpy==6.7.0
Using cached symspellpy-6.7.0-py3-none-any.whl (2.6 MB)
Collecting numpy==1.18.5
Using cached numpy-1.18.5.zip (5.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting flair==0.6.1
Using cached flair-0.6.1-py3-none-any.whl (331 kB)
Collecting keras-self-attention==0.42.0
Using cached keras-self-attention-0.42.0.tar.gz (9.3 kB)
Preparing metadata (setup.py) ... done
ERROR: Could not find a version that satisfies the requirement tensorflow==2.2.0 (from bigbert) (from versions: 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1, 2.5.2, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.1, 2.6.2, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.8.0rc0, 2.8.0rc1)
ERROR: No matching distribution found for tensorflow==2.2.0

This was an environment issue.


Attach commit metadata to the message sent by danger

This makes it easy to see if the range is now what you expect it to be, for example, Danger error'd on this PR - but she shouldn't have. https://github.com/artsy/eigen/pull/1124#issuecomment-180296964 I'd like to know more
A part of #41

  Generated by :no_entry_sign: danger

:shipit:


Adding reset epoch protocol message and support

Stack from ghstack (oldest at bottom):

-> #816

Differential Revision: D40273860

@VitalyFedyunin has imported this pull request.  If you are a Meta employee, you can view this diff on Phabricator.

Is the main difference between ResetEpoch and ResetIterator the fact that the new one will accepts a seed (or other arguments required) to restart the epoch?

That's actually a good question. We might need to change the message name from ResetIterator to InitIterator.


Rustup

@bors r+

:hourglass: Testing commit 55090eec3bcd9d767ef2174a00928bd2c17fd137 with merge 2bd6b4d70416076df9e1a4fa1ac59acfd57cb395...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing 2bd6b4d70416076df9e1a4fa1ac59acfd57cb395 to master...


[BUG] Outdated prior 'constant' set in em_sample.m

Description
When sampling a correlated model with em_sample.m, line 50 sets the prior to 'constant', which is not recognized by bn_dirichlet_prior.m, and results in a terminal exception "Unknown prior of constant, if char expecting prior = 'dbe'".
Reproducibility
Running
em_sample('cor_v2p1.txt')
should result in the exception.
Expectation
Changing line 50 in em_sample.m parms.prior = 'constant' to parms.prior='dbe' should resolve the issue and produce hte expected functionality.
Environment
Debian 10, MATLAB R2021a

This bug was introduced when we first committed the OOP definitions for the EncounterModel superclass and UncorEncounterModel class (commit https://github.com/Airspace-Encounter-Models/em-model-manned-bayes/commit/8bb35720277904e2f2b9d0b075339a0b0efbdb0c). This bug was not caught during testing for commit https://github.com/Airspace-Encounter-Models/em-model-manned-bayes/commit/8bb35720277904e2f2b9d0b075339a0b0efbdb0c because the default functionality in the EncounterModel class sets the prior to 0 and the CorTerminalModel class either sets the prior to 0 or via the setTransitionPriors function.
In commit https://github.com/Airspace-Encounter-Models/em-model-manned-bayes/commit/8bb35720277904e2f2b9d0b075339a0b0efbdb0c, bn_dirichlet_prior was updated to look for 'dbe' instead of 'constant' because we felt 'dbe' was more descriptive than the generic term, 'constant'.
The recommended fix by @reliable-nranganathan should resolve this bug.


working package to read cargo config for private registries

I enabled this to support private registries and to automatically read the cargo config.toml and credentials.toml so it can auth to private repos as well.

@citreae535 any chance you could merge this? I would really like my colleagues to be able to add this through VSCode extensions.


Dual marker analysis fails to start when JLS read still updating

Version: 0.9.2
Platform: All
Description
JLS read can take a long time, as noted in #102.  If you add dual markers and start an analysis, such as Histogram, while the waveform is still updating, the analysis dialog never appears.
Expected Behavior
The analysis dialog should eventually appear.
Developer Notes
The analysis is timing out here.  This timeout kills the reading device permanently without informing the user or attempting to recover.  The 10-second timeout is too fast given that it does not account for the queued updates already processing.
Source: ps_Water on the forum.

Tested as fixed in 1.0.16.  Good luck being fast enough to add markers while the JLS v2 file is loading.


[Question] - Airflow environment variable -> airflow.cfg

Have you been able to add environment variables as describe here :  https://airflow.incubator.apache.org/configuration.html#setting-configuration-options ?
When I add :
AIRFLOW__CORE__SQL_ALCHEMY_CONN in the docker-compose file it gets replaced inside the airflow.cfg
Example :
  webserver:
        image: puckel/docker-airflow:1.8.1-1
        restart: always
        depends_on:
            - postgres
            - redis
        environment:
            - LOAD_EX=y
            - FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
            - EXECUTOR=Celery
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
            - AIRFLOW__CORE__SQL_ALCHEMY_CONN=my_test

but
  webserver:
        image: puckel/docker-airflow:1.8.1-1
        restart: always
        depends_on:
            - postgres
            - redis
        environment:
            - LOAD_EX=y
            - FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
            - EXECUTOR=Celery
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
            - AIRFLOW__SMTP__STMP_HOST=my_smtp_host

the environment variable AIRFLOW__SMTP__STMP_HOST is inside the container but won't get replaced inside the airflow.cfg
I know that one way could be to replace the variable  inside airflow.cfg directly but I wanted to use these environment variables.
Do you think it's because you copy the file instead of letting airflow creating it ?
Thanks for your help.
Best,
Nolan

In fact it will be loaded inside airflow but not replaced in the airflow.cfg config.


Error and Warning log messages displays code instead of verbose word

While displaying error or warnings in the console output log level passed as it is, e.g. as number, while it shold be verbose: 'error', 'warning'.

Resolved by commit 982a41225475804fd25e3227457ff911bbeb0032


Lambda Function Advanced Logging Features Support

Is there a timeline to support the new advanced logging features for Lambda? Specifically, "Customizing Lambda function’s CloudWatch log group".
https://aws.amazon.com/blogs/compute/introducing-advanced-logging-controls-for-aws-lambda-functions/

@Vandita2020 any update on the timeline?


[GOBBLIN-1570] set RETENTION_DATASET_ROOT in CleanableIcebergDataset so that any ret…

…ention job can use this information
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-1570

Description

[x] Here are some details about my PR, including screenshots (if applicable):
Knowing dataset.root of a dataset is a nice-to-have property which can be used by different constructs.

Tests

[X] My PR adds the following unit tests OR does not need testing for this extremely good reason:
trivial changes

Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #3422 (c085b50) into master (384dc09) will decrease coverage by 3.37%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #3422      +/-   ##
============================================
- Coverage     46.52%   43.14%   -3.38%
+ Complexity    10246     2004    -8242
============================================
  Files          2063      401    -1662
  Lines         80475    17275   -63200
  Branches       8989     2121    -6868
============================================
- Hits          37443     7454   -29989
+ Misses        39567     8984   -30583
+ Partials       3465      837    -2628

Impacted Files
Coverage Δ

...a/org/apache/gobblin/util/limiter/NoopLimiter.java
40.00% <0.00%> (-20.00%)
:arrow_down:

...pache/gobblin/cluster/GobblinHelixJobLauncher.java
67.04% <0.00%> (-14.33%)
:arrow_down:

...lin/util/filesystem/FileSystemInstrumentation.java
92.85% <0.00%> (-7.15%)
:arrow_down:

...che/gobblin/cluster/FsJobConfigurationManager.java
66.66% <0.00%> (-4.45%)
:arrow_down:

...in/java/org/apache/gobblin/cluster/HelixUtils.java
28.26% <0.00%> (-3.98%)
:arrow_down:

.../org/apache/gobblin/cluster/GobblinTaskRunner.java
63.46% <0.00%> (-0.33%)
:arrow_down:

...apache/gobblin/salesforce/SalesforceExtractor.java
0.00% <0.00%> (ø)

.../profile/ManagedIcebergCleanableDatasetFinder.java

...gobblin/runtime/api/JobExecutionStateListener.java

.../compaction/verify/CompactionWatermarkChecker.java

... and 1659 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 384dc09...c085b50. Read the comment docs.


Dark theme for line numbers in blame view

A small eye-friendly fix.

Is there a way to cancel CI on force push 😅

Is there a way to cancel CI on force push

yes, but we purposely have that option disabled in drone as it conflicts with minor parts of our workflow, but thankfully we have pretty significant server capacity available for our CI (we can run 10 builds simultaneously).

Codecov Report

Merging #12632 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #12632      +/-   ##
==========================================
- Coverage   43.44%   43.42%   -0.02%
==========================================
  Files         645      645
  Lines       71328    71328
==========================================
- Hits        30985    30977       -8
- Misses      35331    35337       +6
- Partials     5012     5014       +2

Impacted Files
Coverage Δ

modules/queue/workerpool.go
58.77% <0.00%> (-1.23%)
:arrow_down:

modules/log/event.go
56.60% <0.00%> (-0.95%)
:arrow_down:

models/gpg_key.go
54.81% <0.00%> (-0.59%)
:arrow_down:

services/pull/pull.go
41.57% <0.00%> (-0.47%)
:arrow_down:

modules/indexer/stats/db.go
52.17% <0.00%> (ø)

modules/indexer/stats/queue.go
76.47% <0.00%> (+11.76%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e1eee2d...9666e61. Read the comment docs.


flush system

just wondering if maybe adding a flush system when dealing with chemicals to prime up chemicals and flush out chemicals for changing chemicals and cleaning boom for components longevity...

The new controller now includes an option to turn all solenoids on which could be used for flushing. Recirculating booms are quite common for the reasons you mention too.
Is that what you're thinking of?

yeah neat i was thinking maybe keyboard presses but like b button to start flush on timer and e to stop if need

Ah I see, yeh that makes sense. Pretty easy to add if you think it's worthwhile? I can put in a 'flush' option as you mention if you press 'f' on the keyboard. Useful if you have a keyboard out in the field I guess! Will be on the all the controllers as well.


Dependency updates and re-added Web Application Support

I updated the dependencies to ensure the system would build against maven repositories and re-added Web Application support.  The code has been tested and is functional in an Embedded OSGI environment with a Tomcat 7 embed server running from bundles.

Duplicate request, unable to sign previous commits.


left nav still acting funny when clicking around in language modal

see work already done in #3171
clicking in language modal causes highlighting to appear / disappear in left nav
not affecting user's tasks, but is distracting
here's a screen recording:
[Uploading... Screen Recording 2021-10-14 at 4.26.34 PM.mov]

this appears to be fixed as of version 3.12!


add count of total no of words processed

#8 pr has a implementation of creating a summary for the dataset


Delete old additives files

app/src/main/assets/additives_fr.json
app/src/main/assets/additives_en.json

@teolemon  i am working on this


Crossmatch dev

A large commit for updates to the hipscat crossmatching algorithm
-optimized algorithm (faster)
-returns the result as dask.dataframe
-- for further utilizations of the daks.dataframe.api

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@ebc7bf3). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #12   +/-   ##
=======================================
  Coverage        ?   95.66%
=======================================
  Files           ?        9
  Lines           ?      415
  Branches        ?        0
=======================================
  Hits            ?      397
  Misses          ?       18
  Partials        ?        0

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Ruby gets not working

Hi,
I am having trouble running ruby gets and gets.chomp. Upon run, it keeps spinning like it was stuck in a loop. I can run the code just fine from cmd. Any insights?
Thanks.

We don't handle STDIN at all, so it won't work.


Add support for Laravel 8 factories

This PR will allow the package to be used in Laravel 8. The createModels method will now try to create a model using Laravel 8's new factories by first checking if the method factory exists on the model. If it can't find the factory method, it will then check if the global function factory exists. This is necessary because, Laravel 8 has removed this global function and support for it is only available through the laravel/legacy-factories package. If it fails to find both the factory method or factory global function, it will throw an exception. It's possible that this method might not exist on Laravel 8 models if the model isn't using the HasFactory trait.

Hey thanks for contributing!
The code looks good, but I see a couple of improvements before merging:

Can you move the private method to the end of the class? I usually have methods sorted by visibility in classes -> public, protected and private.
Can you rename it to something else? It's misleading that it's named like a singular version of the createModels method when it can also create multiple models. I know, naming is hard 😅 Maybe something like createModelsUsingFactory?


BUG: Filter inputs are erroneously released

The filters that rely on DataObject class, eg. itk::MaskImageFilter corrupt the input image (clears it) despite that both global and filter relative setting to release data is set to false.
Submitted by @urosvovk1. Closes #2023.

Warnings from CircleCI are unrelated:
Examples/Iterators/ImageRandomConstIteratorWithIndex.cxx:137:3: warning: multi-line comment [-Wcomment]


Logs (DEBUG level) go overwhelming when there are some pending requests

Lots of logs were from predicator and some from the core scheduling process. We need to fix this. I can see it goes to hundreds of MBs after running for a while when there are some pending requests.

@yangwwei  perhaps it's related to #54 , scheduling-task logs can be printed repeatedly.

moved to jira:
https://issues.apache.org/jira/browse/YUNIKORN-25


New app: Air Quality

Description
This app allows the user to monitor the air quality in their location with the OpenWeather Air Pollution API:

Whoever is reviewing this app, the production API key is retrieved with the secret module, but you will need to provide a key in the OW_DEV_API_KEY variable if you're testing locally.
You can sign-up and create one in the OpenWeather website (it's free), or hit me up on Discord and I can provide one.
Details about the OpenWeather API and how the app works are provided in the README.md file.
Copilot

🤖 Generated by Copilot at 4d7bbdc
Summary
🌬💡📝

This pull request adds two new apps, Air Quality and LIFX Lights, and updates the README and manifest files for both apps. The Air Quality app shows the current air pollution level and index for a given location, using the OpenWeather Air Pollution API. The LIFX Lights app controls the brightness and color of LIFX smart bulbs, using the LIFX HTTP API. The README files provide documentation and instructions for the apps, and the manifest files provide metadata and configuration for the Tidbyt app store and device.

New and old apps shine
Air Quality and LIFX Lights
Code and docs in sync

Walkthrough

Add a new Air Quality app that displays the current air quality index and pollutants for a given location using the OpenWeather Air Pollution API (link, link, link)
Write the main code for the app in air_quality.star using the Tidbyt SDK and Starlark (link)
Define the schema function to validate and store the app's configuration options, such as the location, units, and API key (link)
Define several helper functions to fetch and process the data from the API, and to render the app's UI with icons, colors, and text (link)
Add the manifest file for the app in manifest.yaml with the app's id, name, summary, description, author, file name, and package name (link)
Add a README file for the app in README.md with an overview of the app's features, API details, configuration options, error handling, and future improvements (link)
Update and improve the existing LIFX Lights app that displays the status and brightness of LIFX smart lights using the LIFX HTTP API (link, link)

Add a README file for the app in README.md with an overview of the app's features, API authentication, rate limiting, configuration options, error handling, and demo/setup mode (link)
Update the manifest file for the app in manifest.yaml with the app's updated summary and description (link)

Will reopen with just the air-quality app.


ui: integrate login with orcid on ui and add local login page

⚠⚠ NEEDS IMPLEMENTING SESSION ID AS AUTHORISATION TOKEN ⚠⚠

ref: cern-sis/issues-inspire#509

Superseded by #3294


Add perimeter stub prior to NGFW cluster onboarding with peering to the shared VPC

Start with rebasing the existing environment
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/97

Get 2 new environments/orgs up
org1 = development of PR changes for peering
org2 = client simulated org with pre-PR main state - that we will apply this 220 PR changes to after prototyping in org1

Fix https://github.com/hashicorp/terraform/issues/31692 first

working

Triage older system for tf version

add to the push cloud build yaml

root_@cloudshell:~/cloudshell_open/pbmm-on-gcp-onboarding (cnpe-cnd-cndev-sbx)$ git diff
diff --git a/modules/cloudbuild/templates/cloudbuild-push-request.yaml b/modules/cloudbuild/templates/cloudbuild-push-request.yaml
index 3495bbe..acc0fbf 100644
--- a/modules/cloudbuild/templates/cloudbuild-push-request.yaml
+++ b/modules/cloudbuild/templates/cloudbuild-push-request.yaml
@@ -14,6 +14,19 @@ steps:
               echo "*************************************************"
               terraform init || exit 1

+- id: 'tf version'
+  name: ${_DEFAULT_REGION}-docker.pkg.dev/${_SEED_PROJECT_ID}/${_GAR_REPOSITORY}/terraform
+  dir: "${_WORKSTREAM_PATH}"
+  entrypoint: 'sh'
+  args:
+  - '-c'
+  - |
+              echo ""
+              echo "*************** TERRAFORM VERSION ******************"
+              echo "******* At environment: ${_WORKSTREAM_PATH} *************"
+              echo "*************************************************"
+              terraform --version || exit 1

results
https://console.cloud.google.com/cloud-build/builds;region=global/1ec94347-7cae-41f6-8d22-0d57878f2a08;step=1?project=cnpe-cnd-cndev-sbx&supportedpurview=project
Terraform v1.0.10
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v3.90.1
+ provider registry.terraform.io/hashicorp/google-beta v4.50.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/random v3.4.3

Your version of Terraform is out of date! The latest version
is 1.3.7. You can update by downloading from https://www.terraform.io/downloads.html

Fix is only required for bootstrap (which runs outside the CB container directly on the gcloud shell (which runs 1.3.7), for all other common/non-prod/prod we can continue to use terraform 1.0.10 and experimental optional attributes

20240406: Closing issue during retrofit/rebase of this TEF V1 based/modified repo to TEF V4 standards
This issue may participate in the LZ refactor after rebase
Query on all issues related to the older V1 version via the tag
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/labels/2024-pre-tef-v4


Add  webots release/version and webots upload fix

Description
This PR tackles the two following issues:

Webots Release will not be compatible with WebotsJS files
Uploads from Webots were not being added to local storage, nor associated with a webots.cloud account if logged in

Release/Version Compatibility
A custom webots.view element is created to view simulations/animations/scenes, and cannot be modified once defined.
When opening a simulation/animation/scene the version is read, if the current WebotsJS module and files are already loaded, we continue.
If not, the previous module is removed and the correct module and files are loaded according to the simulation/animation/scene's release (R2022b, R2023a, etc...), the page is refreshed on load to delete and define a new webots.view element with the new module and files.
Fix Webots Upload Association with Account
A message is sent in the URL and the upload is either associated with the current logged in account, or added to the local storage.
New modal messages have been added and local storage is now updated when deleting a simulation that was found in it.
Additional Modification
Added a new column to each simulation/animation/scene table with the Webots version (Release).

Thank you. Good point about the release tag, modifications have been made.


Leave experimental data alone when cleaning up motion objects

Leave experimental data alone upon motion change
Fixes issue #1168
Brief summary of changes
Testing I've completed
CHANGELOG.md (choose one)

no need to update because...
updated...

@tkuchida Would you have a chance to check this has been fixed in this PR artifact? Thank you

The Issue that was raised in #1168 has been fixed. Minor note: The muscles change color after clicking Poses -> Default (see below). Not sure why… presumably the location of a light is being updated, but it seems like this type of update should occur when the model gets the focus back (i.e. after double-clicking "arm26" in the Navigator) rather than when Poses -> Default is clicked.

Before clicking Poses -> Default
After clicking Poses -> Default

Muscle changing color when state is changed since 4.0 e.g. by manually changing coordinates is expected behavior, otherwise you have inconsistent muscle colors and coordinates.

Thanks for testing @tkuchida


POC provision rework

Codecov Report

Merging #126 (bcec07f) into 2020.12.x (df5ebcf) will increase coverage by 0.02%.
The diff coverage is 34.28%.

@@              Coverage Diff              @@
##           2020.12.x     #126      +/-   ##
=============================================
+ Coverage      38.14%   38.16%   +0.02%
=============================================
  Files            444      444
  Lines          39958    39988      +30
=============================================
+ Hits           15242    15262      +20
- Misses         22738    22747       +9
- Partials        1978     1979       +1

Impacted Files
Coverage Δ

pkg/envoy/resource/compose_page.go
58.53% <0.00%> (-14.20%)
:arrow_down:

pkg/envoy/store/compose_page.go
38.54% <21.05%> (-2.37%)
:arrow_down:

pkg/envoy/store/compose_module.go
54.69% <100.00%> (+0.25%)
:arrow_up:

pkg/envoy/yaml/compose_page.go
76.25% <100.00%> (+1.58%)
:arrow_up:

pkg/handle/handle.go
78.57% <0.00%> (-7.15%)
:arrow_down:

pkg/scheduler/service.go
72.83% <0.00%> (+16.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df5ebcf...bcec07f. Read the comment docs.


JFormMap functionality not working in VR

Apologies if this is a mistake on my end, but I've twice encountered an issue where a script I compiled is functional in SSE using JContainers SE , but not in VR using JContainers VR (both version 4.1.13). My latest instance of the issue is in a very simple script:
Scriptname SynthEBDHeadPartLoaderQuestScript extends Quest

import PSM_SynthEBD

GlobalVariable Property loadingCompleted Auto

Event OnInit()
	debug.Notification("HeadParts OnInit")
	LoadHeadPartDict("OnInit")
EndEvent

Function LoadHeadPartDict(string caller)
	loadingCompleted.SetValue(0)
	int assignmentDict = JValue_readFromFile("Data/SynthEBD/HeadPartDict.json")
	if (assignmentDict)
		debug.Trace("SynthEBD: HeadPart Dict Read")
	else
		debug.Trace("SynthEBD: Failed to read HeadPart Dict")
	endIf
	int count = 0
	int maxCount = JFormMap_count(assignmentDict)
	debug.Trace("SynthEBD: HeadPart Count = " + maxCount as string)
	;more functionality here, but irrelevant for the given issue
EndFunction

The supplied json file exists in both cases, and appears in its entirety as follows:
{
	"__metaInfo": {
		"typeName": "JFormMap"
	},
	"__formData|Skyrim.esm|0x13476": {
		"Hair": "__formData|Skyrim.esm|0x85DC8"
	}
}

The papyrus log from SSE says:
[06/05/2022 - 11:33:08PM] SynthEBD: HeadPart Dict Read
[06/05/2022 - 11:33:08PM] SynthEBD: HeadPart Count = 1

The papyrus log from VR says:
[06/05/2022 - 11:35:45PM] SynthEBD: HeadPart Dict Read
[06/05/2022 - 11:35:45PM] SynthEBD: HeadPart Count = 0

This seems to be identical to the issue I posted about on the SSE Nexus Page on March 27, just in a different context. Am I missing something about how JContainers works, or is the JFormMap functionality not working correctly in the VR build?

I can confirm this. Just a simple dummy test where we write a Form to a file & read it back fails:
; Write to file
int testData = JMap.object()
Form bilegulchForm = Game.GetForm(0x00019269)
If !bilegulchForm
	Debug.MessageBox("Hardcoded bilegulch form invalid")
EndIf
JMap.setForm(testData, "bilegulch", bilegulchForm)
JValue.writeToFile(testData, JContainers.userDirectory() + "testData.txt")

; Read back the very same data JContainers wrote
int loadedTestData = JValue.readFromFile(JContainers.userDirectory() + "testData.txt")
Form loadedBilegulch = JMap.getForm(loadedTestData, "bilegulch")

If !loadedBilegulch ; This path is triggered
	Debug.MessageBox("Failed to load bilegulch form we just saved")
ElseIf loadedBilegulch.GetFormID() != bilegulchForm.GetFormID()
	Debug.MessageBox("Form loaded, but does not match. Loaded="+loadedBilegulch.GetFormId()+" vs Hardcoded="+bilegulchForm.GetFormId())
Else
	Debug.MessageBox("Successfully loaded saved bilegulch Form")
EndIf

The written string to testData.txt is valid
{
  "bilegulch": "__formData|Skyrim.esm|0x19269"
}

and it can be parsed just fine in JContainers SE.

I'm sorry I can't help you here guys. No VR, even less time to try and guess. A good start would be to look here I think: src/skse/skse.cpp:123.

I have ran across this issue when building my list for VR as well, and I think I am onto a fix for it. It appears getting base functionality is as simple as just removing the "#ifndef JC_SKSE_VR" and its related #endif from the function you linked at src/skse/skse.cpp:123 so that the code runs regardless.
Worth noting however that these days we also have VR ESL support which would require extra steps in there and should be accounted for as well, so it would probably be better to add an #else clause instead and handle the logic there to either just do the normal method or handle via VR ESL support if it is installed.
I should be able to get that done and make a PR to resolve this issue this weekend if you want. If so, would you rather the PR go to the latest development and cut a new release from there, or fork off of the v4.1.13-vr tag to minimize changes? Or, since you do not have VR to test would you rather I fork the repo and maintain a separate VR version of this myself so you don't wind up maintaining something you cannot test? I am fine either way.

I should be able to get that done and make a PR to resolve this issue this weekend if you want. If so, would you rather the PR go to the latest development and cut a new release from there, or fork off of the v4.1.13-vr tag to minimize changes? Or, since you do not have VR to test would you rather I fork the repo and maintain a separate VR version of this myself so you don't wind up maintaining something you cannot test? I am fine either way.

For the time being a tested PR would be better, it is easier for the folks to see the releases at one place. For example, Nexus Mods.

I have now made a pull request with my tested and locally proven fixes for this issue.


Build not working with Python 3.13.0b1

Building a wheel for marisa-trie with Python 3.13.0b1 doesn't work, because there are no _PyLong_AsByteArray() and _PyLong_FromByteArray() functions anymore.
The is a CPython issue on this topic (https://github.com/python/cpython/issues/111140), which resulted in alternative public functions being available, so I guess supporting Python 3.13 with marisa-trie should be pretty straight-forward.
Here is a log of a failed build with Python 3.13.0b1: build-log-marisa-trie-python-3.13.txt

Hello @Dunedan!
Thanks for the report, I'll see to add Python 3.13 support 👍🏻

I have the stuff done.
Are you OK if I do not publich Python 3.13 wheels for now? It is not recommanded by cibuildwheel:

This option is provided for testing purposes only. It is not recommended to distribute wheels built when CIBW_PRERELEASE_PYTHONS is set, such as uploading to PyPI. Please do not upload these wheels to PyPI, as they are not guaranteed to work with the final Python release. Once Python is ABI stable and enters the release candidate phase, that version of Python will become available without this flag.

@BoboTiG could you please cut a release? Looks like the 3.13 ABI is now stable.

@iherasymenko I'm on it!

Arf, Python 3.13 wheels worked but not PyPy 3.8, see https://github.com/pytries/marisa-trie/actions/runs/10892762632/job/30226353003
It's getting late here, I'll see later. If anybody can give a hand, I'll be more than happy to see sggestions :)

@BoboTiG could you please cut a release? Looks like the 3.13 ABI is now stable.

Didn't 1.2.0 get already released for Python 3.13 support?

@BoboTiG could you please cut a release? Looks like the 3.13 ABI is now stable.

Didn't 1.2.0 get already released for Python 3.13 support?

Yes, but Python 3.13 wheels were not made available at the time.