[#151266508] Send Rogue campaign signup events to Gambit Conversations

What's this PR do?

Creates gambit-campaign-signup-relay queue which recieves campiang signup messages
Creates gambit-campaign-signup-relay worker which listenes for new messages in gambit-campaign-signup-relay queue and relayes them to Gambit Conversation in specified format

How should this be manually tested?

Sign up for a Thor campaign which is enabled in Gambit Campaigns Staging

What are the relevant tickets?
Pivotal 151266508
Ref https://github.com/DoSomething/gambit-conversations/pull/155

Staging tests are blocked by Rogue Thor maintenance.

Misleading send-message route issue: https://github.com/DoSomething/gambit-conversations/issues/159

Testing is still blocked by broken Ashes -> Rogue integration. Getting "There was an error processing your request" error.

Discovered undesired retry loop for users with no phone number:
at=warning application=blink env=staging code=error_gambit_campaign_signup_relay_response_not_200_retry worker=GambitCampaignSignupRelayWorker request_id=dc052117-3a70-426f-a57f-0b5d0b8220be response_status=500 response_status_text="Internal Server Error" {"message":"The string supplied did not seem to be a phone number"}

Ref https://github.com/DoSomething/blink/pull/147

Ref https://github.com/DoSomething/gambit-conversations/pull/166


autoinject cannot be combined with inject

I'm submitting a bug report

Library Version:
1.2.0

Please tell us about your environment:

Operating System:
Windows 10
Node Version:
6.9.0
NPM Version:
3.10.8
JSPM OR Webpack AND Version
webpack 2.1.0-beta.22
Browser:
Chrome 55
Language:
TypeScript 2.0.3

Current behavior:
Aurelia injects incorrect dependencies when @inject is used in constructor together with @autoinject.
Expected/desired behavior:
@inject should work in combination with @autoinject, especially in case of Element, which needs to have @inject(Element).
I've try reverting back to 1.1.0 and it's working again. I suspect that the error was caused by recent change to https://github.com/aurelia/dependency-injection/blob/master/src/injection.js#L18

For example, with the following TypeScript code:
@autoinject
export class Test {
    constructor(
        private service: MyService,
        @inject(Element) element: Element,
    } { }
}

Version 1.1.0 correctly inject [MyService, Element], while version 1.2.0 incorrectly inject [MyService, Object, Element].

If that worked, it was accidental and probably indeterminate. These two were never intended to be used together.

I'd love to have that working again. Most of the time, everything can be autoinjected, except for Element, which needs @inject(Element). Without being able to use autoinject together with inject, the constructor will be loaded with @inject as soon as Element is needed.
However, I read from the Hub documentation that autoinject does not really needed, if any other attribute is used so that TypeScript will output decorators. Does that mean I can just remove @autoinject and that will work correctly again?

Give it a try :)

So my findings:

@lazy still need to have @autoinject, otherwise, the other dependencies won't get injected correctly.
@inject does not play nice with @autoinject. Remove @autoinject allows @inject to work.
When there're both @inject and @lazy, remove @autoinject.

Sorry, I think I'm lost. When tried with my actual project:

@autonject with @lazy: @lazy(HttpClient) injects function() {}.
@lazy without @autoinject: no inject happens.

I'll try to setup a repo with the issue and update this issue when it's ready.

I'm sorry that I don't know how to create unit tests, can anyone help to create test for the following cases:
import {autoinject} from "aurelia-framework";
class DependencyA { }
class DependencyB { }

@autoinject
class TestBase {
  constructor(private a: DependencyA) { }
}

@autoinject
class TestChild extends TestBase {
  constructor(
    b: DependencyB,
    a: DependencyA,
  } { }
}

Expected: instance of TestBase gets a of type DependencyA. Bug: TestBase gets a with value of DependencyB.
import {lazy} from "aurelia-framework";
import {HttpClient} from "aurelia-http-client";
import {EventAggregator, Subscription} from "aurelia-event-aggregator";

export class MyService {
  constructor(
    private events: EventAggregator,
    @lazy(HttpClient) private lazyHttp: () => HttpClient
  } { }
}

Expected: MyService gets events of type EventAggregator. Bug: MyService get events of type Object.

Does not feel like this should be supported, too much magic for my liking.  How would you, for instance, know which order that they should be injected?
import {inject, autoinject} from 'aurelia-dependency-injection';
import {ExampleTwo} from './example-two';
import {ExampleThree} from './example-three';

@inject(Element)
@autoinject
class MyExample {
  static = [ExampleTwo];
  constructor(exampleTwo, element, exampleThree: ExampleThree) {}
}

Those things are going against each other, just seems kind of odd to support those things together IMHO.

I agree that it does not feel like making sense. Either specify all injection or have all resolved automatically.
Also I want to add that I never had a problem injecting Element with @autoinject - it works as it should.

@ZoolWay is that something new? Problem with Element is a known issue that bugs me so much, see https://github.com/aurelia/dependency-injection/issues/77

@7Pass hmm, strange. I never had the problem that. Using @autoinject with Element since 6/10/2016 as I see in my history. So not sure why Rob says it cannot work. But I am not using in-browser-transpiling and I am targetting ES6 - maybe that is the reason.

FWIW @autoinject() with Element works for me too.
I'm with @PWKad, at first I thought this to be a good idea but now I just don't see how it could work.

I never use the class decorator style @inject(One, Two, Three) now. For me, the moment that @autoinject is no longer an option (i.e. I need @factory, or @newInstance, etc) I just use @inject as a parameter decorator instead:
export class DmaMiniStatus {
  constructor(
    @inject(AwaitAction) public dmaLoadingAction: AwaitAction,
    @inject(AwaitAction) public loggersLoadingAction: AwaitAction,
    @inject(DistrictMeteringAreaRepository) private dmaRepository: DistrictMeteringAreaRepository,
    @inject(LoggerRepository) private loggerRepository: LoggerRepository,
    @newInstance(DistrictMeteringAreaContainer) private dmaContainer: DistrictMeteringAreaContainer,
  ) {
// ...

Until I found this thread some time ago, I didn't realise that using @inject as a parameter decorator was a possibility, so thanks.

Closing issue since it's by design.


Unauthorized logins in Public not displaying error message on login

When trying to login on the Public site with an email that doesn't have an account no error message is being displayed.

QA Notes:

When on the public site login page click "use your password instead"
Enter an email that is not in the system and anything into the password field
On submit make sure that an error message shows.

Tested in staging and the error message is appearing:

I have tested this in production and the error message is properly displaying


Update Pages “home”

Automatically generated by Netlify CMS

👷 Deploy Preview for thunderous-brigadeiros-23902d processing.

Name
Link

🔨 Latest commit
e681efb8c5286264300d1c7377dc57e899a1191c

🔍 Latest deploy log
https://app.netlify.com/sites/thunderous-brigadeiros-23902d/deploys/6331c3627b47da0009964bae


Missing function on new ThinQ refrigerator

Which device do you want to add? Please describe:
I have used this plugin for a few years, but recently had an issue with the fridge and so I have now replaced it with a newer model.
I added it to ThingQ, and added it to HomeKit via this plugin but there are things that are not working.
For example Express Freeze isn't available as an option but I believe this was supported on my old Fridge.
For example in the ThinQ app I can see the status of the ice makers.
It also has functions to enable "Smart Learning" to adjust things based on usage patterns.  The ThinQ app lets you turn these features on or off.
(The fridge has UV sterilisation for the water dispenser - it would be good if this could be activated from HomeKit but it appears that this is not exposed in the ThinQ App)
But the Fridge also has other features like
Describe the solution you'd like:

It would be good to have more  features available!
Device Data:

[04/12/2023, 4:23:45 pm] [homebridge-lg-thinq] [request] get https://kic-service.lgthinq.com:46030/v1/service/homes/160709041675589464
[04/12/2023, 4:23:45 pm] [homebridge-lg-thinq] Device [Impressive Cold Monolith]:  54b473e5-bd3a-1bc4-8c35-a436c70c82c6: Impressive Cold Monolith (REFRIGERATOR 2REF11EICSP_4)
[04/12/2023, 4:23:45 pm] [homebridge-lg-thinq] {"modelAppType":"NUTS","brandType":"common","deviceId":"54b473e5-bd3a-1bc4-8c35-a436c70c82c6","deviceType":101,"modelName":"2REF11EICSP_4","subModelNm":null,"sensorType":null,"alias":"Impressive Cold Monolith","deviceCode":"KI0102","networkType":"02","tftYn":"N","guideTypeYn":"Y","guideType":"TYPE1","pccModelYn":"N","autoOrderYn":"N","drServiceYn":"N","ssid":"punch","timezoneCode":"Australia/Darwin","timezoneCodeAlias":"AUS/Darwin","sdsGuide":"{\"deviceCode\":\"KI01\"}","newRegYn":"Y","remoteControlType":null,"fareTarget":null,"area":"13713","sleep":null,"deviceState":"E","rmsClientId":null,"regDtUtc":"20231204015017000","regIndex":0,"blackboxYn":"Y","groupableYn":"N","controllableYn":"N","combinedProductYn":"N","masterYn":"Y","snapshot":{"fwUpgradeInfo":{"upgSched":{"upgUtc":"0","cmd":"none"}},"static":{"deviceType":"101","countryCode":"AU"},"meta":{"allDeviceInfoUpdate":false,"messageId":"8IroWONTTQWq5Fw1fVbrAg"},"packageId":"4346100143460901|1","mid":0,"online":true,"refState":{"dispenserCapacity":255,"dispenserUnit":"IGNORE","freezerTemp":4,"iceMaker2Status":"IGNORE","sabbathMode":"OFF","tempUnit":"CELSIUS","ecoFriendly":"IGNORE","activeSaving":"IGNORE","voiceMode":"IGNORE","fridgeStatus":"STABLIZING","smartSavingRun":"STOP","lightDimMode":"IGNORE","atLeastOneDoorOpen":"CLOSE","expressMode":"OFF","freshAirFilter":"AUTO","convertibleTemp":255,"waterFilter1RemainP":100,"waterFilter":"0_MONTH","waterFilter2RemainP":255,"waterFilter3RemainP":255,"freshAirFilterRemainP":100,"nightAntiGlareMode":"OFF","dispenserMode":"NOT_DEFINE_VALUE value:255","displayLock":"IGNORE","nightQuietMode":"OFF","expressFridge":"IGNORE","iceMaker1Status":"ICE_MAKING","convertibleStatus":"IGNORE","selfCareMode":"IGNORE","drawerMode":"IGNORE","foodPoisonIndex":58,"fridgeTemp":5,"iceMaker3Status":"ICE_MAKING","monDataNumber":2,"pantryMode":"IGNORE","craftIceMode":"6_ICE","dualFridgeMode":"IGNORE","monStatus":"NORMAL","smartSavingMode":"SMARTGRID_DD_ON","freezerStatus":"STABLIZING","smartCareV2":"ON"},"timestamp":1701672028961},"manufacture":{"inventoryOrg":"CP3","macAddress":"A4:36:C7:0C:82:C6","manufactureModel":"GC-X22FTQXL.AMCRGAP","manufacturedAt":"2023-08-17T18:08:29+00:00","registeredAt":"2023-08-18T05:13:29.401844+00:00","salesModel":"GF-V570MBLC.AMCRGAP","serialNo":"308TRSE1W286"},"online":true,"platformType":"thinq2","homeDeviceOrder":-1,"roomDeviceOrder":-1,"ownershipYn":"N","modelJsonVer":"1.03","modelJsonUri":"https://objectcontent.lgthinq.com/bbd03ab5-37cf-4679-8eee-68fbd7ae8faf?hdnts=exp=1763884709~hmac=c5622b5c1b8a8fe10c4953e0638fee07b958df74a96e79faa64397aca6fef051","appModuleVer":"303.35","appModuleUri":"https://objectcontent.lgthinq.com/d9e9cbd3-9734-4d6b-88d4-003d35cd5ca1?hdnts=exp=1763600486~hmac=a76bfbf93a566c767708fc4f50876f0101b926420b1f37c012d4dc951fbf8c99","appRestartYn":"Y","appModuleSize":"8344898","langPackProductTypeVer":"72.8","langPackProductTypeUri":"https://objectcontent.lgthinq.com/fc087856-c408-45e4-b84e-13268df2fd77?hdnts=exp=1764487294~hmac=40fa32c77b16b1f9ae03adeb272920f871dd6a911074aa431d89e4853a1c87f4","langPackModelVer":null,"langPackModelUri":null,"roomId":"160422194461218976","fwInfoList":[{"checksum":"01075684","order":2,"partNumber":"SAA43460901"},{"checksum":"00000443","order":1,"partNumber":"SAA43461001"}],"modemInfo":{"appVersion":"clip_hna_v1.9.183","modelName":"2REF11EICSP_4","modemType":"RTK_RTL8711am","oneshot":"y","ruleEngine":"y","size":1572864},"existsEntryPopup":"N","fwVer":null,"modemVer":"clip_hna_v1.9.183","subDeviceCount":0,"firebaseLogKey":null,"cardType":"Small","cardControl":"Base","detailDeviceCode":null,"upgradableYn":"N","autoFwDownloadYn":"N","homeMovableYn":"Y","protocolVersion":null,"btAddress":null,"isTLV":null,"jsonList":null,"mdFotaYn":"Y","demandType":"RTK_RTL8711am"}
[04/12/2023, 4:23:45 pm] [homebridge-lg-thinq] [Impressive Cold Monolith] Setting up device!
[04/12/2023, 4:23:45 pm] [homebridge-lg-thinq] Adding new accessory: 54b473e5-bd3a-1bc4-8c35-a436c70c82c6: Impressive Cold Monolith (REFRIGERATOR 2REF11EICSP_4)

Any news here?  Its a pity this fridge doesn't at least support the features that were on my old fridge (eg express freeze)

For example Express Freeze isn't available as an option but I believe this was supported on my old Fridge.

you can enable this feature in plugin setting, I think it's called "express mode"

For example in the ThinQ app I can see the status of the ice makers.

I'll add it in future update

It also has functions to enable "Smart Learning" to adjust things based on usage patterns. The ThinQ app lets you turn these features on or off.

I don't see anything related this feature in API responsed, I got smartSavingRun, smartCareV2, smartSavingMode flag, not sure if it's "Smart Learning" feature
you need enable homebridge debug mode, then try collect some logging when this feature toggled, let see which one is changed

(The fridge has UV sterilisation for the water dispenser - it would be good if this could be activated from HomeKit but it appears that this is not exposed in the ThinQ App)

if you can't control it in LG app, this plugin can't too

Thanks for the update!
On 13 Feb 2024 at 16:26:53, nVuln @.***> wrote:

For example Express Freeze isn't available as an option but I believe this
was supported on my old Fridge.
you can enable this feature in plugin setting, I think it's called
"express mode"

There is no such setting for my new fridge.  (I still have my old fridge in
the config file and I can see the setting there)
But I can’t see a switch in the home app.
Looking at the plugin config I can see the old one is identified as
"type": "REFRIGERATOR" but the new one isn’t.
Here is the new one:
{
"id": "54b473e5-bd3a-1bc4-8c35-a436c70c82c6",
"name": "New Fridge"
}

For example in the ThinQ app I can see the status of the ice makers.
I'll add it in future update

Cool!

It also has functions to enable "Smart Learning" to adjust things based on
usage patterns. The ThinQ app lets you turn these features on or off.
I don't see anything related this feature in API responsed, I got
smartSavingRun, smartCareV2, smartSavingMode flag, not sure if it's
"Smart Learning" feature
you need enable homebridge debug mode, then try collect some logging when
this feature toggled, let see which one is changed

I think it covers a few different features.  When you look on the app it
comes up with the following:
If you select any of those it shows when various things are scheduled, eg:
Anyway, here is the debug info.
I turned Smart Learning off and see:
[Impressive Cold Monolith] Received snapshot:
{"fwUpgradeInfo":{"upgSched":{"upgUtc":"0","cmd":"none"}},"static":{"deviceType":"101","countryCode":"AU"},"meta":{"allDeviceInfoUpdate":false,"messageId":"fcB-bnHnRbO9YZWUE4-vAA"},"packageId":"4346100143460901|1","mid":1227476204,"online":true,"refState":{"dispenserCapacity":255,"dispenserUnit":"IGNORE","freezerTemp":5,"iceMaker2Status":"IGNORE","sabbathMode":"OFF","tempUnit":"CELSIUS","ecoFriendly":"IGNORE","activeSaving":"IGNORE","voiceMode":"IGNORE","fridgeStatus":"GOOD","smartSavingRun":"STOP","lightDimMode":"IGNORE","atLeastOneDoorOpen":"CLOSE","expressMode":"OFF","freshAirFilter":"AUTO","convertibleTemp":255,"waterFilter1RemainP":63,"waterFilter":"2_MONTH","waterFilter2RemainP":255,"waterFilter3RemainP":255,"freshAirFilterRemainP":81,"nightAntiGlareMode":"CUSTOM","dispenserMode":"NOT_DEFINE_VALUE
value:255","displayLock":"IGNORE","nightQuietMode":"OFF","expressFridge":"IGNORE","iceMaker1Status":"ICE_MAKING","convertibleStatus":"IGNORE","selfCareMode":"IGNORE","drawerMode":"IGNORE","foodPoisonIndex":75,"fridgeTemp":3,"iceMaker3Status":"ICE_MAKING","monDataNumber":20,"pantryMode":"IGNORE","craftIceMode":"6_ICE","dualFridgeMode":"IGNORE","monStatus":"NORMAL","smartSavingMode":"SMARTGRID_DD_ON","freezerStatus":"GOOD","smartCareV2":"OFF"},"timestamp":1707810516892}
Turning it back on (and restarting home bridge) I get
Impressive Cold Monolith] Received snapshot:
{"fwUpgradeInfo":{"upgSched":{"upgUtc":"0","cmd":"none"}},"static":{"deviceType":"101","countryCode":"AU"},"meta":{"allDeviceInfoUpdate":false,"messageId":"MXo8btR9TwW9WNaF1VGVSg"},"packageId":"4346100143460901|1","mid":0,"online":true,"refState":{"dispenserCapacity":255,"dispenserUnit":"IGNORE","freezerTemp":5,"iceMaker2Status":"IGNORE","sabbathMode":"OFF","tempUnit":"CELSIUS","ecoFriendly":"IGNORE","activeSaving":"IGNORE","voiceMode":"IGNORE","fridgeStatus":"GOOD","smartSavingRun":"STOP","lightDimMode":"IGNORE","atLeastOneDoorOpen":"CLOSE","expressMode":"OFF","freshAirFilter":"AUTO","convertibleTemp":255,"waterFilter1RemainP":63,"waterFilter":"2_MONTH","waterFilter2RemainP":255,"waterFilter3RemainP":255,"freshAirFilterRemainP":81,"nightAntiGlareMode":"CUSTOM","dispenserMode":"NOT_DEFINE_VALUE
value:255","displayLock":"IGNORE","nightQuietMode":"OFF","expressFridge":"IGNORE","iceMaker1Status":"ICE_MAKING","convertibleStatus":"IGNORE","selfCareMode":"IGNORE","drawerMode":"IGNORE","foodPoisonIndex":74,"fridgeTemp":3,"iceMaker3Status":"ICE_MAKING","monDataNumber":20,"pantryMode":"IGNORE","craftIceMode":"6_ICE","dualFridgeMode":"IGNORE","monStatus":"NORMAL","smartSavingMode":"SMARTGRID_DD_ON","freezerStatus":"GOOD","smartCareV2":"ON"},"timestamp":1707810666550}
There was also a longer message:
{"modelAppType":"NUTS","brandType":"common","deviceId":"54b473e5-bd3a-1bc4-8c35-a436c70c82c6","deviceType":101,"modelName":"2REF11EICSP_4","subModelNm":null,"sensorType":null,"alias":"Impressive
Cold
Monolith","deviceCode":"KI0102","networkType":"02","tftYn":"N","guideTypeYn":"Y","guideType":"TYPE1","pccModelYn":"N","autoOrderYn":"N","drServiceYn":"N","ssid":"punch","timezoneCode":"Australia/Darwin","timezoneCodeAlias":"AUS/Darwin","sdsGuide":"{"deviceCode":"KI01"}","newRegYn":"N","remoteControlType":null,"fareTarget":null,"area":"13713","sleep":null,"deviceState":"E","rmsClientId":null,"regDtUtc":"20231204015017000","regIndex":0,"blackboxYn":"Y","groupableYn":"N","controllableYn":"N","combinedProductYn":"N","masterYn":"Y","snapshot":{"fwUpgradeInfo":{"upgSched":{"upgUtc":"0","cmd":"none"}},"static":{"deviceType":"101","countryCode":"AU"},"meta":{"allDeviceInfoUpdate":false,"messageId":"MXo8btR9TwW9WNaF1VGVSg"},"packageId":"4346100143460901|1","mid":0,"online":true,"refState":{"dispenserCapacity":255,"dispenserUnit":"IGNORE","freezerTemp":5,"iceMaker2Status":"IGNORE","sabbathMode":"OFF","tempUnit":"CELSIUS","ecoFriendly":"IGNORE","activeSaving":"IGNORE","voiceMode":"IGNORE","fridgeStatus":"GOOD","smartSavingRun":"STOP","lightDimMode":"IGNORE","atLeastOneDoorOpen":"CLOSE","expressMode":"OFF","freshAirFilter":"AUTO","convertibleTemp":255,"waterFilter1RemainP":63,"waterFilter":"2_MONTH","waterFilter2RemainP":255,"waterFilter3RemainP":255,"freshAirFilterRemainP":81,"nightAntiGlareMode":"CUSTOM","dispenserMode":"NOT_DEFINE_VALUE
value:255","displayLock":"IGNORE","nightQuietMode":"OFF","expressFridge":"IGNORE","iceMaker1Status":"ICE_MAKING","convertibleStatus":"IGNORE","selfCareMode":"IGNORE","drawerMode":"IGNORE","foodPoisonIndex":74,"fridgeTemp":3,"iceMaker3Status":"ICE_MAKING","monDataNumber":20,"pantryMode":"IGNORE","craftIceMode":"6_ICE","dualFridgeMode":"IGNORE","monStatus":"NORMAL","smartSavingMode":"SMARTGRID_DD_ON","freezerStatus":"GOOD","smartCareV2":"ON"},"timestamp":1707810666550},"manufacture":{"inventoryOrg":"CP3","macAddress":"A4:36:C7:0C:82:C6","manufactureModel":"GC-X22FTQXL.AMCRGAP","manufacturedAt":"2023-08-17T18:08:29+00:00","registeredAt":"2023-08-18T05:13:29.401844+00:00","salesModel":"GF-V570MBLC.AMCRGAP","serialNo":"308TRSE1W286"},"online":true,"platformType":"thinq2","homeDeviceOrder":-1,"roomDeviceOrder":-1,"ownershipYn":"N","modelJsonVer":"1.03","modelJsonUri":"
https://objectcontent.lgthinq.com/bbd03ab5-37cf-4679-8eee-68fbd7ae8faf?hdnts=exp=1763884709~hmac=c5622b5c1b8a8fe10c4953e0638fee07b958df74a96e79faa64397aca6fef051
","appModuleVer":"303.35","appModuleUri":"
https://objectcontent.lgthinq.com/d9e9cbd3-9734-4d6b-88d4-003d35cd5ca1?hdnts=exp=1763600486~hmac=a76bfbf93a566c767708fc4f50876f0101b926420b1f37c012d4dc951fbf8c99
","appRestartYn":"Y","appModuleSize":"8344898","langPackProductTypeVer":"76.4","langPackProductTypeUri":"
https://objectcontent.lgthinq.com/28beb7d1-d90e-420b-b708-6c889a5d4166?hdnts=exp=1770449317~hmac=4bd353c5861e15e97c76203673c1d5a39ef30484df74cefe6295537b7ea16a22
","langPackModelVer":null,"langPackModelUri":null,"roomId":"160422194461218976","fwInfoList":[{"checksum":"01075684","order":2,"partNumber":"SAA43460901"},{"checksum":"00000443","order":1,"partNumber":"SAA43461001"}],"modemInfo":{"appVersion":"clip_hna_v1.9.198","modelName":"2REF11EICSP_4","modemType":"RTK_RTL8711am","oneshot":"y","ruleEngine":"y","size":1572864},"modemConfig":{"online_check":"y"},"existsEntryPopup":"N","fwVer":null,"modemVer":"clip_hna_v1.9.183","subDeviceCount":0,"firebaseLogKey":null,"cardType":"Small","cardControl":"Base","detailDeviceCode":null,"upgradableYn":"N","autoFwDownloadYn":"N","homeMovableYn":"Y","protocolVersion":null,"btAddress":null,"isTLV":null,"jsonList":null,"mdFotaYn":"Y","demandType":"RTK_RTL8711am"}

(The fridge has UV sterilisation for the water dispenser - it would be
good if this could be activated from HomeKit but it appears that this is
not exposed in the ThinQ App)
if you can't control it in LG app, this plugin can't too

I suspect that is true.  Although the debug info makes me think there might
be features that are not present in all markets (eg Sabbath Mode!)
—

Reply to this email directly, view it on GitHub
https://github.com/nVuln/homebridge-lg-thinq/issues/267#issuecomment-1940526683,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJP3T2G54DRDGEBTTD2OE3YTMFDLAVCNFSM6AAAAABAFPLXM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBQGUZDMNRYGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

There is no such setting for my new fridge. (I still have my old fridge in the config file and I can see the setting there) But I can’t see a switch in the home app. Looking at the plugin config I can see the old one is identified as "type": "REFRIGERATOR" but the new one isn’t.

did you used homebridge UI?
try add config below
{
            "id": "54b473e5-bd3a-1bc4-8c35-a436c70c82c6",
            "name": "New Fridge",
            "ref_express_fridge": true,
            "ref_express_freezer": true
}

this config will enable 2 more button, let see which one is correct for you

I suspect that is true.  Although the debug info makes me think there might
be features that are not present in all markets (eg Sabbath Mode!)

try debug it like you did with "Smart Learning", maybe it's present as different name in API

On 13 Feb 2024 at 21:51:13, nVuln @.***> wrote:

There is no such setting for my new fridge. (I still have my old fridge in
the config file and I can see the setting there) But I can’t see a switch
in the home app. Looking at the plugin config I can see the old one is
identified as "type": "REFRIGERATOR" but the new one isn’t.
did you used homebridge UI?

I tried to use home bridge UI, but no settings appeared in the UI until I
edited the config.json to include  "type": "REFRIGERATOR".  Is this
supposed to be detected automatically?

try add config below
{
"id": "54b473e5-bd3a-1bc4-8c35-a436c70c82c6",
"name": "New Fridge",
"ref_express_fridge": true,
"ref_express_freezer": true
}
I have now done this - but as mentioned added to set the type as well.

this config will enable 2 more button, let see which one is correct for you

Express freezer is the right one.  (I can turn express Fridge on and off
but it doesn’t seem to do anything).
I suspect that is true. Although the debug info makes me think there might

be features that are not present in all markets (eg Sabbath Mode!)
try debug it like you did with "Smart Learning", maybe it's present as
different name in API

Its not clear how much value some of that would be.
There are many more things in the thinq app - ice modes, detection the ice
dispenser is full, indications that the fridge/freezer is temperature is
being maintained, indications that food safety has been kept etc.  I guess
it could be useful to expose some of that to HomeKit.

I—
Reply to this email directly, view it on GitHub
https://github.com/nVuln/homebridge-lg-thinq/issues/267#issuecomment-1941393022,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJP3TYPCUV5I4V6AI3PQYLYTNLDTAVCNFSM6AAAAABAFPLXM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBRGM4TGMBSGI
.
You are receiving this because you authored the thread.Message ID:
@.***>


About the image upload function improvement

After uploading the image, you need to copy the image address to the posting content area, so it is too troublesome, can upload directly in the content area?

It seems impossible without JavaScript


cardano-api: Redo unix-2.8 fixes.

My fix had to be reverted.  https://github.com/input-output-hk/cardano-api/pull/139

Done, but did not work 🤷


Add dhclient.conf

Null out dhclient.conf's set host-name

What's the reason for doing that? Also, would this be different for different distributions of Linux?


Fix for objectives

I'm working on the objectives for a backend branch.
Please take a look at my sqrt fix. I don't know why lower bound is epsilon.
Also, i have two questions:
How can i test both backends in the same time? It seems like it's a problem for a current architecture.
What do you think about my approach to test objectives? I can replace this with static and precomputed examples.

Please resubmit when ready to merge.

@fchollet Can you answer my questions, please?


Using transformations in objects' names

Hello!
First of all, thank you for sharing your work. The whole idea of generating Helm charts on the fly according to a single values file is fantastic.
I'm trying to use HULL in my work, and I'd like to ask you for some advice.
Is it possible to assign an object's name by a variable set by a user? Something kind of that:
hull:
  objects:
    secret:
      _HT^something.secretName:
<...skipped...>
something:
  secretName: my-secret

The user sets something.secretName, and my chart creates a Secret with this name prefixed by the chart and release names.
Obviously, the straightforward way I showed in my example won't work. So is there a right way to achieve this?
Thanks in advance for any hints.
P.S. Merry Christmas! 🎄

Ah, I see: metadataNameOverride is what I actually need. Sorry for the disturbance.

Yes, the intention behind metadataNameOverride was exactly this use case. And no disturbance at all, if you have further questions or problems please share them.
Thanks for the kind words, have a nice christmas time!


Remove duplicate ODP cal invite and update text

[ ] Remove calendar dup - @mindthegab - I sent you a request via google calendar, to own the event ; could you please accept?
[ ] Update invite text to link to odp.finos.org pages (ie Team, Project Collaboration, etc) - @aitana16 is the owner of the ODP meeting event
[ ] Update list of attendees, remove people who didn't attend for a long time

hi @maoo which invite text should I update? The one of the invite that has 74 guests or the one of the invite that has 41 guests? Thanks.

The one of 41 guests, as it's owned by you and not by Gab.
TY!
https://www.finos.org/Maurizio Pillitu | DevOps Director | FINOS
https://finos.github.io/ https://twitter.com/finosfoundation
https://www.facebook.com/finosfoundation
https://linkedin.com/company/finosfoundation
FINOS joined the Linux Foundation
https://t.sidekickopen80.com/s1t/c/5/f18dQhb0S7lM8dDMPbW2n0x6l2B9nMJN7t5X-FfhMynW4Xrjwx1pNBMbW56dPQv6dxTzn102?te=W3R5hFj4cm2zwW4mKLS-3M3CVyW4cg8wC3NC11MW3K9dy93M3CVFW3K2y-X43VqYYW1GHF1749JGg5W3M66lv3H3bCtW43Tw4643RxqhW1GDK984hNYSsW43Xx5z3zhs7SW41p0ZJ1GzR8WW3zdYS73zdYS7W3z8myw3ZVdlSW4fGvkg3KbsMRW43VrFJ41YsqyW3FcKLt4cfLf7W41PG5N3SXXpmW4cJ2Vx3SYLML0&si=7000000001133467&pi=b32bc874-b20e-4f7b-f526-56ce6dc06f6c
to
accelerate even further the growth open source in financial services.. Get
involved
https://t.sidekickopen80.com/s1t/c/5/f18dQhb0S7lM8dDMPbW2n0x6l2B9nMJN7t5X-FfhMynW4Xrjwx1pNBMbW56dPQv6dxTzn102?te=W3R5hFj4cm2zwW4mKLS-3M3CVyW4cg8wC3NC0dSW4fdh0F4kHSCY38QZ2&si=7000000001133467&pi=b32bc874-b20e-4f7b-f526-56ce6dc06f6c
today
and become a FINOS Member
https://t.sidekickopen80.com/s1t/c/5/f18dQhb0S7lM8dDMPbW2n0x6l2B9nMJN7t5X-FfhMynW4Xrjwx1pNBMbW56dPQv6dxTzn102?te=W3R5hFj4cm2zwW4mKLS-3M3CVyW4cg8wC3NB_Q7W3Fbt6S1Gy1HNn3K6KcQ263&si=7000000001133467&pi=b32bc874-b20e-4f7b-f526-56ce6dc06f6c
to
contribute building  a true open financial stack and ecosystem.
On Thu, Jul 2, 2020 at 12:13 AM aitana16<EMAIL_ADDRESS>wrote:

hi @maoo https://github.com/maoo which invite text should I update? The
one of the invite that has 74 guests or the one of the invite that has 41
guests? Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/finos/open-developer-platform/issues/144#issuecomment-652672203,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACP45NW36ROBXZCNQUVRFDRZOYKFANCNFSM4ONWP5KQ
.

thanks @maoo I've deleted the duplicate calendar invite (the one Gab had created) and updated the other calendar invite with the information we agreed to over symphony and google docs. I'm unable to check the box, feel free to check it for me. Thanks =)

TY @aitana16 ! I still see the duplicate in my calendar

Not sure if you have visibility on the 2 events, I asked @mindthegab to help, as he's the owner of the event that must be removed.
Let's check on Monday and also clean up the list of participants.
TY!

Hi Mao, I removed/ deleted the duplicate invite (the one going out of Gab's calendar) yesterday. I cannot see any duplicates on the FINOS projects calendar or on my personal calendar either..It may be a ghost event. Happy to discuss further on a call on Monday.


What about native dependencies where the .node binary is not the only architecture-specific file?

Related to https://github.com/lovell/sharp/issues/3622 - Sharp on macOS requires both a .node and and a .dylib binary but @electron/universal only seems to manage the .node files, ignoring the necessity of the .dylib.
I'm not certain this is an @electron/universal issue, but after speaking with Sharp's creator and doing some more thinking, I believe that this is the case. Is that something that can be managed by this project? Or am I misunderstanding the problem?

Based on this comment, I feel even more confident that this could work if the .dylib files were copied for both architectures, as the .node files already are.

Anyone have any idea about this? Sharp is a wildly popular library and I think @electron/universal is a step away from properly supporting it - but I'd love to hear from a contributor or maintainer and find out if I'm misunderstanding. Otherwise maybe we could make a lot more apps universal.

Friendly bump. This seems to be the only blocker for us to make universal Electron builds.

@MarshallOfSound do you have any thoughts on this? It seems like an oversight in how universal packages are built that they don't include all of the required native files for both architectures. Instead the bundle only includes a copy for one architecture, so the package just crashes on the other.

I encountered the same problem and tried many solutions but none of them worked.

@Slapbox We try to automatically merge any file that looks like a AppFileType.MACHO, not sure why it wouldn't be picking up a dylib correctly as it does for instance work for the dylibs embedded in electron.
Do you have a repro case?

Thanks for your reply @MarshallOfSound. I wonder if it's related to the fact that Sharp (iirc) only downloads the .dylib files for the given platform when it installs itself. I'm not sure exactly how Electron incorporates .dylibs so maybe my question is silly.
I don't have a repro case handy but I'll try to make one in the next week or two.

@mushan0x0 thanks for your comment in another thread. If you need this sooner and can create a repro that would be a big help to getting this resolved faster (and to me personally) - all the investigative work on this set me back on some other tasks.
My plan to produce a repro is to modify this boilerplate: https://github.com/electron-react-boilerplate/electron-react-boilerplate/

Install Sharp, and upgrade Electron-Builder dep to ^24.0.0
Include sharp in the main process
Configure the build process to do universal macOS builds here (either inline or via config file)
Build on macOS
When you run you should produce the error message we've both been encountering (unless I've missed a step)

@MarshallOfSound one other thing that might be of interest while waiting on the repro - from this comment: https://github.com/electron-userland/electron-builder/issues/7512#issuecomment-1500613835
Is this how you'd expect the file structure to look? It seems unexpected to me that there's no darwin-arm64 or equivalent. Do you mean that the darwin-x64 contents should automatically be made universal by @electron/universal despite the naming convention?

@Slapbox I had a chance to look into this, it looks like this is super specific to sharp in that the native module build process does not generate that dylib, instead it's generated at install time by the install/libvips script.
Although @electron/rebuild will correctly rebuild the native module it has no idea about the existence of that dylib build process.  You'll have to somehow tell electron-builder (no idea how, we don't support that tool) to run the libvips script for each arch during the package phase so that each version of the packaged app has the right dylib.
Closing this out as it's a build script issue not an issue with the universal library


Use LH node base image

Currently we are using a node vanilla base image in our docker UT image https://github.com/littlehorse-enterprises/lh-user-tasks/blob/main/Dockerfile#L1
To enable TLS we should use nginx, we have create a base image for that https://github.com/littlehorse-enterprises/alpine-nginx-nodejs
nginx configuration here https://github.com/littlehorse-enterprises/alpine-nginx-nodejs/blob/main/nginx.conf
Goal
Adapt our UT image to use the LH base image

The problem with this is we need the base image to act in a way that its agnostic whether we use a mono repo or not.
We can try to fit the current Next Build to fit into the base image mold but I would not recommend that and instead recommend having a more agnostic image.

Well it is the one that the dashboard is using https://github.com/littlehorse-enterprises/littlehorse/blob/master/docker/dashboard/Dockerfile#L1
If it is well parameterized it is good, a lot of services (keycloak for example) provide embedded TLS solutions. Let's have further discussions with @mijailr

@sauljabin The dashboard doesn't use npm workspaces so when building with next it does not treat it like a monorepo.
Because we have a package that is being used in the dashboard next is building it in 2 subdirectories instead of the root directory.
Lets connect with @mijailr for a possible solution.

@sauljabin The dashboard doesn't use npm workspaces so when building with next it does not treat it like a monorepo.
Because we have a package that is being used in the dashboard next is building it in 2 subdirectories instead of the root directory.
Lets connect with @mijailr for a possible solution.

Got it, in that case we can copy most of the configurations into this repo and customize it for it


chore(operator, yaml): add openebs operator 2.1.0

add openebs operator 2.1.0
add cstor operator 2.1.0 and dependents
add zfs localPV 1.0.0 operator
add openebs operator lite
add openebs operator for ARM
add openebs operator for suse caas platform

Signed-off-by: Akhil Mohan<EMAIL_ADDRESS>

Added operator yamls for 2.1.0-RC2

Added operator yamls for 2.1.0


minor change for validating input types

ASSERT_TRUE(st.IsOK());

another UT failure from here.

Refers to: onnxruntime/test/framework/inference_session_test.cc:1271 in d1bf1c6. [](commit_id = d1bf1c6e4c2f31564f1a5e734c000dcf8a04747d, deletion_comment = False)

The perf numbers are, for bingads classifier model, baseline is ~279.617us, now it's ~268.683us.


Your Workspaces menu not working and keeps flashing

Describe the bug
The Your Workspaces tab on the settings menu keeps refreshing without any action.
To Reproduce
Steps to reproduce the behaviour:

Go to 'Settings'
Click on 'Your Workspaces'
See error

Expected behaviour
A clear and concise description of what you expected to happen.
Screenshots
Video: https://d.pr/v/Y03nik
Desktop (please complete the following information):

OS: macOS 10.14
Franz Version 5.1.0

I expect the same issue but I'm able to see my workspaces after 3-4 seconds after this "flashing".

This issue should have been resolved - what's the status for you @rgllm and @promarcel ?

Thanks @DominikGuzei. I will check as soon as possible.


Generic types for es6 classes and getters

I am trying to annotate a getter on an es6 class with a generic type, but I can't seem to get it working.
A simplified example of the problem is:
// ==ClosureCompiler==
// @compilation_level ADVANCED_OPTIMIZATIONS
// @output_file_name default.js
// @language_out ES5
// ==/ClosureCompiler==

/** @template T */
class X {

  /** @param {T} x */
  constructor(x) {
    this.x = x;
  }

  /** @return {T} */
  get foo() {
    return this.x;
  }
}

Produces:
JSC_TYPE_PARSE_ERROR: Bad type annotation. Unknown type T at line 9 character 15
  /** @return {T} */

which is the annotation on the foo getter (the constructor seems to work fine)
(Checked with https://closure-compiler.appspot.com)

I am happy to look into it if anyone has a couple of pointers where a good place to start would be.

I think I promised someone I would fix this a while back, and then it turned out to be harder than I thought. I believe the @template annotation on the class gets copied as you'd expect, but the problem is that the getter lives outside of the class in the transpiled output:
/** @constructor @template T */
function X() {}

Object.defineProperties(X.prototype, {
  foo: {
    /** @return {T} */    // <- The compiler doesn't know what you mean by "T" here.
    get: function() { return this.x; }
  }
});

I suspect this will ultimately be fixed by having the typechecker understand ES6 classes (including getters and setters) directly instead of typechecking the output of the 6-to-5 conversion :(

I just stumbled upon this while converting my library to ES2015. What's the status of this? Is the typechecker still checking the ES5 output?

@b-strauss
The short answer is "yes, the type checker still only sees ES5 code".
Efforts are underway to change this.
Many optimization passes have now been updated to understand ES6 code, and it is now possible to request ES6 output from the compiler.
However, the type checker is one of the hardest parts to change and we're part way through switching from OTI (old type inference) to NTI (new type inference), which complicates things.

This is fixed now


Review optimisation for BYTE, SHL, SHR and SAR

Reference to this: https://github.com/imapp-pl/gas-cost-estimator/blob/master/docs/report_stage_iii.md#analysis-4

hmm
want to take a look

done in #1418


[server-wallet] Add a 2x Wallet createChannel test

Preparatory work towards ledger-funding. Test can cover some or all funding strategies.

Test fails because 'status' does not update as expected
Apparently no difference in behaviour when passing a different funding strategy
test is (almost) self contained with the help of a bit of execSync and a new DBAdmin().migrate() helper. It just needs a postgres server running
generated #2659

Added 0329ee7 based on this Notion strategy. Doesn't include rejecting objectives, and just uses an in-memory objectives store, but gets the test to pass.

Going to close this in favour of new smaller PRs


Formatting in container on save

Hello, I have followed the guide for the installation and updated my settings.json. When I run VSCode in a container, my files aren't format on save. When I use Ctrl+P it says there is multiples formatters for python files, and when I choose black it format my code without issues. Hower there is no logs when I save without using Ctrl+P format document.
If I use a virtualenv and use it on my local machine instead of the docker container, the files are formatted correctly.
If I black with the python extension it works even in the container

@GwendalFernet Can you verify if these settings were present in the Container settings?
  "[python]": {
    "editor.defaultFormatter": "ms-python.black-formatter",
    "editor.formatOnSave": true
  }

If those settings are added correctly it should work.

The one in container overrides the User settings, and the one in .vscode folder in your workspace overrides any container, user or machine level settings.


delayed evaluation of annotations causes type mismatches

Current behavior
importing annotations from __future__ causes externally provided nodes to fail type validation.
Stack Traces
Traceback (most recent call last):
  File "c:\Users\gravesee\projects\dsbsvc-dsgtools-modgov\scratch.py", line 21, in <module>
    dr.execute(["dep"])
  File "c:\Users\gravesee\projects\dsbsvc-dsgtools-modgov\.venv\Lib\site-packages\hamilton\driver.py", line 261, in execute
    raise e
  File "c:\Users\gravesee\projects\dsbsvc-dsgtools-modgov\.venv\Lib\site-packages\hamilton\driver.py", line 254, in execute
    outputs = self.raw_execute(_final_vars, overrides, display_graph, inputs=inputs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\gravesee\projects\dsbsvc-dsgtools-modgov\.venv\Lib\site-packages\hamilton\driver.py", line 354, in raw_execute
    self.validate_inputs(
  File "c:\Users\gravesee\projects\dsbsvc-dsgtools-modgov\.venv\Lib\site-packages\hamilton\driver.py", line 225, in validate_inputs
    raise ValueError(error_str)
ValueError: 1 errors encountered:
  Error: Type requirement mismatch. Expected dep:str got hello instead.

Screenshots
(If applicable)
Steps to replicate behavior
from __future__ import annotations

def func1(dep: str) -> str:
    return dep

def func2(dep: str) -> str:
    return dep

if __name__ == "__main__":
    import __main__ as main

    from hamilton.driver import Driver

    config = {
        "dep": "hello"
    }

    dr = Driver(config, main)
    dr.execute(["dep"])

Library & System Information
python 3.11.1
sf-hamilton==1.18.0
Windows 11
Expected behavior
The above code works without any issues by commenting out the from __future__ import annotations line.
Additional context
I prefer some type annotation patterns that aren't supported in earlier versions of python. Yet my team needs to target python 3.9+. I imported annotations to make use of the nicer type union syntax.

Nice find. Also want to point out that I really like the import __main__ as main trick.
Pretty sure the fix is easy but will take a bit of time -- we need to use typing.get_type_hints(fn) instead of inspect.signature(fn) -- i think this should work in all cases, but the devil is in the details.
Happy to take a stab at this.
Out of my own curiosity -- are you making use of delayed type annotations much?

OK -- have a quick POC that works with your example. Mind giving the branch a spin? https://github.com/DAGWorks-Inc/hamilton/pull/92.
Something like: pip install git+https://github.com/dagworks-inc/hamilton.git@issue_91 will get it installed.
For productionalizing this I need to go through and see where else we're using the fn signature...

~~Out of my own curiosity -- are you making use of delayed type annotations much? ~~ Ahh just saw the additional context

My team mostly uses them for classmethods or builder-pattern type methods that return self. We use mypy in pre-commit with fairly conservative settings and it complains about methods without return types. We could use a string as the return type, but the delayed evaluation is supposed to land in python eventually -- although it seems like that's not the case any more.

OK -- have a quick POC that works with your example. Mind giving the branch a spin? #92.
Something like: pip install git+https://github.com/dagworks-inc/hamilton.git@issue_91 will get it installed.
For productionalizing this I'll add testing with the __future__ import as well.

Yes! The code that was previously failing is now running without issue. This behavior was a real adventure to nail down.

OK -- have a quick POC that works with your example. Mind giving the branch a spin? #92.
Something like: pip install git+https://github.com/dagworks-inc/hamilton.git@issue_91 will get it installed.
For productionalizing this I'll add testing with the __future__ import as well.

Yes! The code that was previously failing is now running without issue. This behavior was a real adventure to nail down.

Awesome! Will be getting this out into production shortly -- sorry you had to spend time debugging -- going to be adding some tests to make sure our suite works with the new annotations feature (need to double-check a few extensions.

OK, this has been merged and we'll be planning to release. Thanks for finding the bug! https://github.com/DAGWorks-Inc/hamilton/pull/92

@gravesee would you mind validating things work with pip install sf-hamilton==1.19.0rc0 please?

sf-hamilton==1.19.0rc0

Works on 1.19.0rc0 as well!


Invalid type: ignore style when checking with mypy

The jsonschema module has 3 type: ignore comments that are not in PEP484 format. This results in the following error when running mypy where referencing has been imported:
$ mypy my_package
/usr/local/lib/python3.8/dist-packages/referencing/jsonschema.py:432: error: Invalid "type: ignore" comment
/usr/local/lib/python3.8/dist-packages/referencing/jsonschema.py:474: error: Invalid "type: ignore" comment
/usr/local/lib/python3.8/dist-packages/referencing/jsonschema.py:505: error: Invalid "type: ignore" comment
Found 3 errors in 1 file (errors prevented further checking)

The error produced is the same as reported here. However, newer versions of mypy do not fix the issue, as the style is not correct.
These can't be ignored with the mypy ignore_errors = True flag, because these are fatal errors that do not allow mypy to run over the rest of the codebase.
From what I can tell, the error messages just need an extra # after the error code to allow mypy to pass

Hi there -- pyright (which is what's used here in the repo) has no issue with these, which is why CI passes -- I'm happy to merge a PR to change the syntax, though ideally one that also then adds a minimal mypy setup which would have failed for this, otherwise it's hard to promise it won't happen again.

It looks like this specific error only occurs in mypy<0.990, so in my setup updating mypy should be fine. I don't think it's worth a mypy check against v0.982 just for this case. But there are of course other blocking mypy errors, so it might be good to make this change, and add the minimal mypy setup just to be safe? What do you think @Julian?

I opened the PR anyway 🙂


Standardize a way of dealing with multiple devices of the same class

As per this discussion on Zulip, we need to implement a concise way of dealing with multiple devices of the same class. Right now, we only support a single device per class, which severely limits our abilities to build more complex systems.
Inside Linux, USB devices are organized as a tree, for example:
~ $ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/12p, 480M
    |__ Port 5: Dev 2, If 1, Class=Video, Driver=uvcvideo, 480M
    |__ Port 5: Dev 2, If 0, Class=Video, Driver=uvcvideo, 480M
    |__ Port 6: Dev 3, If 0, Class=Wireless, Driver=btusb, 12M
    |__ Port 6: Dev 3, If 1, Class=Wireless, Driver=btusb, 12M
    |__ Port 7: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 12M

This provides a useful starting point as this addressing system is easily scalable for daisy-chaining USB hubs. Moreover, most drivers provide a by-path symlink already, allowing us to connect directly to the device at a given port.
For instance, one of the video device listed above can be found at /dev/v4l/by-path/pci-0000:00:14.0-usb-0:5:1.0-video, where pci-0000:00:14.0 is the location of the root hub (integrated to the motherboard), usb-0 is the hub itself 5 is the port and 1.0 is the interface.
Therefore, my suggestion is that we provide a similar addressing scheme for our users. This would require rewriting each class and defining a user-friendly way to present the addresses. My suggestion is following the standard Linux convention, but stripping it from redundant parts like the PCI address. For instance, the device above could be reached by usb-0:5.

[ ] Rewrite digitalRelay
[ ] Rewrite network
[ ] Rewrite power
[ ] Rewrite SD mux
[ ] Rewrite serial
[ ] Rewrite video

Related #29


Houdini: Lower level publishing concepts

Changelog Description
This PR implements some ideas for "lower level publishing" in Houdini.
As part of the Ynput Houdini Workgroup sessions I developed this quick prototype to expose a way to batch publish and ingest files. Consider it more of an exploration of what's possible then a "drop it in production now" ready-to-go solution.

TOPs/PDG example on accessing AYON api for folders, tasks, workfiles and publish the workfiles in background houdini processes (similar to legacy/old style publishing of workfiles)
TOPs/PDG example on low level ingesting of publishes using an in-memory dynamic Creator.
Generic ROP publishing (Creator to publish from any ROP node)

Note that this is Houdini-tested but the "lower level dynamic" publishing is logic that is not Houdini only.
Additional info
Explainer
Yes, this requires some explanation. Here you go.
https://github.com/ynput/ayon-core/assets/2439881/5f767493-10bd-41f6-b54f-89313a96da33
What I forgot to add is that it currently still relies on detecting what the output files are for a ROP node based on a "file" parm that is often unique in Houdini per node. If anyone knows a way to just query the expected output files for a ROP node (similar to what PDG/TOPs seems to do I'd love to know!) but otherwise we'll just need to expand that list.
However, I also played with the idea of having "custom file list" attributes on the Node that when enabled could override the "Collector" logic and would instead use that list of files as the publish instance's files. So that e.g. one instance could also publish multiple representations. For that, @MustafaJafar did this lovely non-functional 'prototype' but it does get the idea across:

Demo scene file
The demo scene file:
ynts_char_hero_pdg_v012.zip
Testing notes:

Check out the branch
Check out the explainer video
Test the demo scene file
Comment with great ideas on how to improve

Instances are not unique error in demo file
So in my recording I hit this issue with the low-level PDG example:
instance product names ['/HoudiniTesting > pointcacheMain'] are not unique. Please remove or rename duplicates

With that demo scene you'll hit the same error because those PDG nodes cook in-process so the logic runs in the current houdini process. Since it triggers a regular CreateContext logic it also collects whatever I happen to have in my Houdini scene as instances, like the pointcacheMain ROP I generated during the demo itself - oops.
So basically even without the dynamic instances the CreateContext already initializes to:
Collected create context with 3 instances
- pointcacheMain (pointcache)
- modelTess (pointcache)
- workfilePdg (workfile)

Deleting those nodes for model and pointcache from the workfile will fix that and allow you to test it further. Note that the workfile will always be collected (but the instance could technically be deactivated through the instance, etc.) Anyway, something to be aware of. (You could technically also open the publisher UI, toggle off all instances, click save and close it again so that they are off by default when the CreateContext finds them)
The better solution would be to run the cook for those PDG/TOPs nodes Cook (Out-of-process) but you'll need to make sure that those run in the same outer process so they have access to the exact python objects that is the CreateContext instance itself.

Also noticed the dynamic logic doesn't like it yet if the ROP fetch uses a ROP that generates a single file - so that should be "todo" to fix as well, shouldn't be too hard.

Here's my demo(s)!
https://github.com/ynput/ayon-core/assets/4348536/5b2bece1-8242-441e-9a4d-3010c10420b6
I had to re-record the end because my microphone did a prank on me too
https://github.com/ynput/ayon-core/assets/4348536/8841a171-a5dc-4d82-98ee-03108643e502

Here's my demo(s)!

I'm now realizing that I forgot to touch on what the AX Publisher Submitter ROP node does. It's basically a very simple approach to traverse through the whole upstream node graph (taking into account bypassed nodes and switches) and run the publish submission on the nodes that have the publish folder.
And I'm not sure if I mentioned this point on my second recording but the main thing here is that this pipeline I showed is production ready, all the tools that artists need to render/cache in the farm and publish all types are there. This is what an MVP (minimum viable product) is, after having this, then you can start adding complexity, but we should never start from a complex system and work all the way down IMO

Hey,
I was testing the batch creator and I find it very cool.
I wrote my notes about it in this gist https://gist.github.com/MustafaJafar/bd2a388e4a6aa3613d64a186ebb6660c
as well as I extended the batch creator in PR https://github.com/BigRoy/ayon-core/pull/6 to support multiple representation.
Here's screen shot after seeing "Publish succeeded." for the first time.

This shows that published pointcache has multiple representation.

[wait a demo about it]

Just want to say thanks @fabiaserra @MustafaJafar - those are great details!
I like:

The variant (or in your case direct product name) being $OS by default. - this PR currently defaults to just Main.
The comment field being available on the ROP nodes - which this PR lacks.
The product types being an enum with predefined standards per node type.
The publish logic being available even if just creating a regular Houdini ROP node.
The flipbook tool!

I dislike (some from the demos, some are comments on how things currently work in AYON):

Them not being detected as regular publish instances (which could greatly borrow from this PRs logic)
The need to go directly through completely separate publishing/ingesting logic. What I'd want to come out of this PR and/or concept is have a low-level API that would make that technically feasible (and easy!) out of the box. Whether that'd still go through pyblish or not is up for discussion, but just having something almost as simple as a publish(product_name, representations) functionality exposed would be great.
The validators are, like @fabiaserra described, too strict (or just plain wrong) and block just having this be a generic "publish me anything" workflow. We should find a way to have those validators be very targeted to only a very specific workflow, instead of just "all pointcache product types" from Houdini. Maybe even link it to a particular creator (or its identifier).
I dislike the flipbook itself seeming quite 'unlinked' to a particular publish of e.g. a pointcache (I'd preferably have that tightly coupled together so we can pipeline-wise make assumptions like "this review belongs to this published pointcache")
I want a headless publish to preferably also be able to generate a json publish report that can be loaded in the publish report viewer. However, @iLLiCiTiT told me that logic is still tightly coupled to the UI currently but shouldn't need to be. Once that's lower-level available on any publish then we can have say a PDG publish, or whatever publish log, be shared for someone on another machine (like a dev) to debug what went wrong with that particular publish.

Also got some comments from Alexandru Preoteasa (Static VFX):

I just took a look at the Lower level publishing concepts you put up on github. Looks interesting and seems like the AYON api gives you quite a decent amout of control. One thing I'm really interested in TOPs rather than batch publish and ingest is actually being able to automate processes that have to run in a chain and have all the reviews published along the way.

Which made me wonder @fabiaserra did you end up doing that in your setups as well? I suppose just a regular OpenGL node could be used in the /out network with review product type and due to how dependencies work with the publishing with Deadline that would automatically become like a dependent job, etc.

Deadline submissions, and then publishing.
@fabiaserra  What's a bit unclear from your video is that you're using the regular deadline submission. Does that automatically also publish the output in a separate deadline job that gets created by the deadline submitter because you "hacked that into it"? Or how does that work? It seems you're basically submitting it to Deadline on its own - basically close to no AYON related logic to that, but then when/how does that publish through AYON?

Flipbook tool
Also really like the Flipbook tool you showed in the Part2 video at 03:30 with how easy it is to create the preview, check it and then publish it directly there.

Thank you @BigRoy for summarizing your highlights and trying to transform these into action items! Let me try respond to some of your points.

Them not being detected as regular publish instances (which could greatly borrow from this PRs logic)

I agree that would be nice to close the bridge with making use of the existing publish dialog and plugin system (although quite a lot of Houdini TDs would still prefer to submit their publishes through the node graph like we do with the AX Publisher Submitter and not a separate dialog) but the main point IMO is whether the AYON publish framework is fit for accommodating this workflow or we are trying to use a screwdriver as a hammer (as brought up a bit ago by Max on this post https://community.ynput.io/t/to-pyblish-or-not-to-pyblish/932).

The need to go directly through completely separate publishing/ingesting logic. What I'd want to come out of this PR and/or concept is have a low-level API that would make that technically feasible (and easy!) out of the box. Whether that'd still go through pyblish or not is up for discussion, but just having something almost as simple as a publish(product_name, representations) functionality exposed would be great.

I agree and I guess this touches my prior point. We created our own separate API that abstracts the process of creating the .json file to run headless publishes and I keep using on all the tools that I have been building for our needs and I find it extremely useful. This stems from the fact that when I needed to create these tools I found it very hard to dissect the publish framework to run it in a simple manner from other places. If I had understood then how to do it through the system, I wouldn't have created this separate abstraction. I think the batch publishing introduced here kind of goes into this direction but I still find it too verbose and rigid, like mentioned by @BigRoy here https://gist.github.com/MustafaJafar/bd2a388e4a6aa3613d64a186ebb6660c?permalink_comment_id=5067094#gistcomment-5067094

The validators are, like @fabiaserra described, too strict (or just plain wrong) and block just having this be a generic "publish me anything" workflow. We should find a way to have those validators be very targeted to only a very specific workflow, instead of just "all pointcache product types" from Houdini. Maybe even link it to a particular creator (or its identifier). What are your thoughts @iLLiCiTiT @antirotor ?

Yeah, unless the validators are 100% reliable and cover all use cases (quite impossible in reality, specially when AYON tries to fit a lot of workflows from different studios), it adds extra blockers to using the system. All validators should be optional and the publish dialog shouldn't show you ALL the plugins that exist in the system because it's very hard to know which plugins are actually required for that publish instance and they add too much noise to make the dialog actually useful to read.

I dislike the flipbook itself seeming quite 'unlinked' to a particular publish of e.g. a pointcache (I'd preferably have that tightly coupled together so we can pipeline-wise make assumptions like "this review belongs to this published pointcache")

Yeah definitely, although that would be a very easy addition to the abstracted publish API to collect inputs on submission (reusing the same logic of the existing AYON plugin for doing that).

So as @MustafaJafar stated:

What about converting the product type Houdini parameter in generic instances to a drop down menu.

Yes! But preferably in a way that we don't need to go create complex creators all over the place, etc.

Please!

Which made me wonder @fabiaserra did you end up doing that in your setups as well? I suppose just a regular OpenGL node could be used in the /out network with review product type and due to how dependencies work with the publishing with Deadline that would automatically become like a dependent job, etc.

In my publish abstraction module, for image type sequences (i.e. render, review...) I spin up a Deadline task as a pre-dependency to the AYON/OP publish that runs a Deadline Nuke plugin that creates a .mov from the image sequence through a Nuke template script that we define (globally or per show if we want to set any overrides). That generated video is what gets uploaded into Shotgrid as the video representation of the publish version (by adding it into the expected representations of the .json that we use to do the headless publish) so we would very rarely need a separate review instance. Using OpenGL nodes as part of the publish process would be possible to add on the TOP dependency graph but that would just be only for flipbook type of reviews and it would have some limitations -> running the OpenGL ROP in the farm requires a GPU on the worker.

Deadline submissions, and then publishing.
@fabiaserra What's a bit unclear from your video is that you're using the regular deadline submission. Does that automatically also publish the output in a separate deadline job that gets created by the deadline submitter because you "hacked that into it"? Or how does that work? It seems you're basically submitting it to Deadline on its own - basically close to no AYON related logic to that, but then when/how does that publish through AYON?

Correct, we have two different Deadline submission entry points. I haven't hacked the vanilla Houdini Deadline submission yet to make it understand AYON and automatically create publish tasks as dependency tasks, although it would be certainly possible and not too hard to do. However, we currently are on the philosophy that artists need to always validate the things that get published so we wouldn't want to run the publish tasks on the same render submission. The two entry points are:

Deadline ROP submitter to render and orchestrate dependencies of the node graph in the farm (with only a few modifications to support USD Render and inject the OP/AYON env vars to extract the AYON environment on runtime) with very little saying from AYON.
AX Publisher Submitter to ONLY submit the publish of the previously generated outputs (it runs some simple pre-validations to make sure the outputs exist on disk) as separate Deadline tasks running the AYON plugin. For each node that's publishable we get a task such as this one (on this case it also automatically contains a Nuke task as a dependency that generates the .mov to upload as the review representation):

Flipbook tool
Also really like the Flipbook tool you showed in the Part2 video at 03:30 with how easy it is to create the preview, check it and then publish it directly there.

The tool is basically this right now, the code is not the cleanest as it was mostly just a copy paste from an old tool we had in our legacy pipeline to have an MVP they can start using already:
import os
import logging

from qtpy import QtWidgets, QtGui
import hou

from ayon_core.lib import path_tools
from ayon_core.modules.deadline.lib import publish

log = logging.getLogger(__name__)

class FlipbookDialog(QtWidgets.QDialog):
    def __init__(self, parent=None):
        QtWidgets.QDialog.__init__(self, parent)

        self.scene_viewer = hou.ui.paneTabOfType(hou.paneTabType.SceneViewer)

        # other properties
        self.setWindowTitle("Flipbook")

        # define general layout
        layout = QtWidgets.QVBoxLayout()
        groupLayout = QtWidgets.QVBoxLayout()

        # output toggles
        self.outputToMplay = QtWidgets.QCheckBox("MPlay Output", self)
        self.outputToMplay.setChecked(True)

        self.beautyPassOnly = QtWidgets.QCheckBox("Beauty Pass", self)
        self.useMotionblur = QtWidgets.QCheckBox("Motion Blur", self)

        # description widget
        self.descriptionLabel = QtWidgets.QLabel("Description")
        self.description = QtWidgets.QLineEdit()

        resolution = self.get_default_resolution()

        # resolution sub-widgets x
        self.resolutionX = QtWidgets.QWidget()
        resolutionXLayout = QtWidgets.QVBoxLayout()
        self.resolutionXLabel = QtWidgets.QLabel("Width")
        self.resolutionXLine = QtWidgets.QLineEdit(resolution[0])
        resolutionXLayout.addWidget(self.resolutionXLabel)
        resolutionXLayout.addWidget(self.resolutionXLine)
        self.resolutionX.setLayout(resolutionXLayout)

        # resolution sub-widgets y
        self.resolutionY = QtWidgets.QWidget()
        resolutionYLayout = QtWidgets.QVBoxLayout()
        self.resolutionYLabel = QtWidgets.QLabel("Height")
        self.resolutionYLine = QtWidgets.QLineEdit(resolution[1])
        resolutionYLayout.addWidget(self.resolutionYLabel)
        resolutionYLayout.addWidget(self.resolutionYLine)
        self.resolutionY.setLayout(resolutionYLayout)

        output_path = self.get_output_path()
        self.outputLabel = QtWidgets.QLabel(
            f"Flipbooking to: {output_path}"
        )

        # resolution group
        self.resolutionGroup = QtWidgets.QGroupBox("Resolution")
        resolutionGroupLayout = QtWidgets.QHBoxLayout()
        resolutionGroupLayout.addWidget(self.resolutionX)
        resolutionGroupLayout.addWidget(self.resolutionY)
        self.resolutionGroup.setLayout(resolutionGroupLayout)

        # frame range widget
        self.frameRange = QtWidgets.QGroupBox("Frame range")
        frameRangeGroupLayout = QtWidgets.QHBoxLayout()

        # frame range start sub-widget
        self.frameRangeStart = QtWidgets.QWidget()
        frameRangeStartLayout = QtWidgets.QVBoxLayout()
        self.frameRangeStartLabel = QtWidgets.QLabel("Start")
        self.frameRangeStartLine = QtWidgets.QLineEdit("$RFSTART")

        frameRangeStartLayout.addWidget(self.frameRangeStartLabel)
        frameRangeStartLayout.addWidget(self.frameRangeStartLine)
        self.frameRangeStart.setLayout(frameRangeStartLayout)
        frameRangeGroupLayout.addWidget(self.frameRangeStart)

        # frame range end sub-widget
        self.frameRangeEnd = QtWidgets.QWidget()
        frameRangeEndLayout = QtWidgets.QVBoxLayout()
        self.frameRangeEndLabel = QtWidgets.QLabel("End")
        self.frameRangeEndLine = QtWidgets.QLineEdit("$RFEND")

        frameRangeEndLayout.addWidget(self.frameRangeEndLabel)
        frameRangeEndLayout.addWidget(self.frameRangeEndLine)
        self.frameRangeEnd.setLayout(frameRangeEndLayout)
        frameRangeGroupLayout.addWidget(self.frameRangeEnd)

        self.frameRange.setLayout(frameRangeGroupLayout)

        # copy to path widget
        self.copyPathButton = QtWidgets.QPushButton("Copy Path to Clipboard")

        # options group
        self.optionsGroup = QtWidgets.QGroupBox("Flipbook options")
        groupLayout.addWidget(self.outputToMplay)
        groupLayout.addWidget(self.beautyPassOnly)
        groupLayout.addWidget(self.useMotionblur)
        groupLayout.addWidget(self.copyPathButton)
        self.optionsGroup.setLayout(groupLayout)

        # button box buttons
        self.cancelButton = QtWidgets.QPushButton("Cancel")
        self.startButton = QtWidgets.QPushButton("Start Flipbook")
        self.publishButton = QtWidgets.QPushButton("Submit to Publish")
        self.publishButton.setEnabled(
            os.path.exists(hou.expandString(output_path))
        )

        # lower right button box
        buttonBox = QtWidgets.QDialogButtonBox()
        buttonBox.addButton(self.startButton, QtWidgets.QDialogButtonBox.ActionRole)
        buttonBox.addButton(self.cancelButton, QtWidgets.QDialogButtonBox.ActionRole)
        buttonBox.addButton(self.publishButton, QtWidgets.QDialogButtonBox.ActionRole)

        # widgets additions
        layout.addWidget(self.outputLabel)
        layout.addWidget(self.descriptionLabel)
        layout.addWidget(self.description)
        layout.addWidget(self.frameRange)
        layout.addWidget(self.resolutionGroup)
        layout.addWidget(self.optionsGroup)
        layout.addWidget(buttonBox)

        # connect button functionality
        self.cancelButton.clicked.connect(self.close_window)
        self.startButton.clicked.connect(self.start_flipbook)
        self.publishButton.clicked.connect(self.submit_to_publish)
        self.copyPathButton.clicked.connect(self.copy_path_to_clipboard)
        self.description.textChanged.connect(self.update_output_path)

        # finally, set layout
        self.setLayout(layout)

    def close_window(self):
        self.close()

    def update_output_path(self):
        output_path = self.get_output_path()
        self.outputLabel.setText(f"Flipbooking to: {output_path}")
        self.publishButton.setEnabled(
            os.path.exists(hou.expandString(output_path))
        )

    # get a flipbook settings object and return with given inputs
    def get_flipbook_settings(self, input_settings):
        settings = self.scene_viewer.flipbookSettings().stash()
        log.info("Using '%s' object", settings)

        # standard settings
        settings.outputToMPlay(input_settings["mplay"])
        settings.output(input_settings["output"])
        settings.useResolution(True)
        settings.resolution(input_settings["resolution"])
        settings.cropOutMaskOverlay(True)
        settings.frameRange(input_settings["frameRange"])
        settings.beautyPassOnly(input_settings["beautyPass"])
        settings.antialias(hou.flipbookAntialias.HighQuality)
        settings.sessionLabel(input_settings["sessionLabel"])
        settings.useMotionBlur(input_settings["motionBlur"])

        return settings

    def get_output_path(self, expand=False):
        description = self.description.text().replace(" ", "_")
        path = "$HIP/flipbook/$HIPNAME/flipbook{}.$F4.jpg".format(
            f"_{description}" if description else ""
        )
        if expand:
            path = hou.expandString(path)

        return path

    def get_default_resolution(self):
        cam = self.scene_viewer.curViewport().camera()
        if not cam:  # Use the main render_cam if no viewport cam
            cam = hou.node("/obj/render_cam")

        if cam:
            x = cam.parm("resx").eval()
            y = float(cam.parm("resy").eval())
            pixel_ratio = cam.parm("aspect").eval()
            return (x, int(y / pixel_ratio))

        return ("$RESX", "$RESY")

    def start_flipbook(self):

        inputSettings = {}

        # validation of inputs
        inputSettings["frameRange"] = self.get_frame_range()
        inputSettings["resolution"] = self.get_resolution()
        inputSettings["mplay"] = self.outputToMplay.isChecked()
        inputSettings["beautyPass"] = self.beautyPassOnly.isChecked()
        inputSettings["motionBlur"] = self.useMotionblur.isChecked()

        outputPath = self.get_output_path()
        inputSettings["output"] = outputPath
        inputSettings["sessionLabel"] = outputPath

        log.info("Using the following settings, %s", inputSettings)

        base_dir = os.path.dirname(hou.expandString(outputPath))
        if not os.path.exists(base_dir):
            os.makedirs(base_dir)

        # retrieve full settings object
        settings = self.get_flipbook_settings(inputSettings)

        # run the actual flipbook
        try:
            with hou.InterruptableOperation(
                "Flipbooking",
                long_operation_name="Creating a flipbook",
                open_interrupt_dialog=True,
            ) as operation:
                operation.updateLongProgress(0.25, "Starting Flipbook")
                hou.SceneViewer.flipbook(self.scene_viewer, settings=settings)
                operation.updateLongProgress(1, "Flipbook successful")
                # self.close_window()

        except Exception as e:
            log.error("Oops, something went wrong!")
            log.error(e)
            return

        self.publishButton.setEnabled(
            os.path.exists(hou.expandString(outputPath))
        )

    def submit_to_publish(self):
        output_path = self.get_output_path(expand=True)
        product_name = os.path.basename(output_path).split(".")[0]
        # Add task name suffix to product name
        product_name = f"{product_name}_{os.getenv('AYON_TASK_NAME')}"

        if not os.path.exists(output_path):
            hou.ui.displayMessage(
                f"Flipbook path {output_path} does not exist, generate it first.",
                title="Path does not exist",
                severity=hou.severityType.Error,
            )

        button_idx, values = hou.ui.readMultiInput(
            "Publish Input",
            input_labels=("Comment", "Version (optional)"),
            buttons=("Submit", "Cancel"),
            default_choice=0,
            close_choice=1,
            initial_contents=(
                "",
                path_tools.get_version_from_path(hou.hipFile.basename())
            )
        )

        if button_idx:
            return

        comment, version = values

        publish_data = {"out_colorspace": "rec709"}

        if comment:
            publish_data["comment"] = comment

        if version:
            publish_data["version"] = int(version)

        message, success = publish.publish_version(
            os.getenv("AYON_PROJECT_NAME"),
            os.getenv("AYON_FOLDER_PATH"),
            os.getenv("AYON_TASK_NAME"),
            "review",
            product_name,
            {"jpg": output_path},
            publish_data=publish_data,
            overwrite_version=True if values[1] else False,
        )
        if success:
            hou.ui.displayMessage(
                message,
                title="Submission successful",
                severity=hou.severityType.Message,
            )
        else:
            hou.ui.displayMessage(
                message,
                title="Submission error",
                severity=hou.severityType.Error,
            )

    # copyPathButton callback
    # copy the output path to the clipboard
    def copy_path_to_clipboard(self):
        path = self.get_output_path(expand=True)
        log.info("Copying path to clipboard: %s", path)
        QtGui.QGuiApplication.clipboard().setText(path)

    def get_frame_range(self):
        return (
            int(hou.expandString(self.frameRangeStartLine.text())),
            int(hou.expandString(self.frameRangeEndLine.text())),
        )

    def get_resolution(self):
        return (
            int(hou.expandString(self.resolutionXLine.text())),
            int(hou.expandString(self.resolutionYLine.text())),
        )

def show_dialog(parent):
    dialog = FlipbookDialog(parent)
    dialog.show()

Hey!
We have redesigned the LabsKarma node for rendering according to our studio requirements. it's really a good one!
If the ayon publishing part was added to it. The renders can be published without any hiccups. the current ayon creating karma node maynot be a best solution. I know lot of studios have there own file cache and render HDAs. I love the concept of adding AYON parameters into the node and publish through that node. Lot of studios are migrating to solaris Karma mostly due to materialx support. I think it will be beneficial. Do checkout the LabsKarma ayon can add parameters and publish through it, it would be awesome!

Also product type of the fetch can be anything eg:render/pointcache, instead string parameter, we can add menu?

Yes. That'd be similar to what @fabiaserra showed in his demo I suppose.

I was just exploring this PR and I have some feedback on this: adding a fetch is really a cool idea, maybe we should whitelist it for the publishing

Fetch is a tricky beast to tackle - this is also visible in e.g. deadline's source code. Technically it's not the fetch generating anything. Also fetch the way you seem to want to use brings up the debate whether that single fetch node would be one instance, or multiples with different product. Since technically you could fetch any files, any types and any upstream hierarchy which may generate many files?
Could you describe your use case - how do you want to use fetch and why?

@krishna8008 I was able to make LabsKarma publishable using this PR and adding few more tweaks.
Steps:

Use the generic creator while setting product type to karma_rop.
Change the file method to explicit
add the following lines just below this

    elif node_type == "labs::karma::2.0":
        return node.parm("file")

render the frames
set render target to use existing frames (local)
publish

I admit it looks weird in the loader in Houdini. but hey, the files are where they should be and they are tracked by AYON.

Also, this doesn't support separated AOVs because the current karma_rop product was built based on the karma rop which doesn't support AOVs.

Well, your example exposed many things in the Houdini addon like

It's rigid.
The generic creator was designed to enable ROP nodes to behave as though they were produced by various creators from Ayon.  This ensures that validators and collectors are retained without loss.

tbh, sometimes I imagine generic creator as fabia mentioned in his demo and as the following mockup screenshot. where you specify the product type and list of files to include but this leads to losing the validators and other plugins.
I believe many studios can't abandon because these plugins combined forms their pipeline.

Thank You @MustafaJafar, its working great for LabsKarma, we can publish renders through it now!
But I was just wondering Its not working for the custom LabsKarma node! Please have a look at the below image

We saved as a new copy and added deadline and other features inside. I also matched the type of the HDA in lib.py. still its not working.
Every other studio/artist will have there own HDA. It would be great, if Ayon tools can be designed in a way that it can fit.

Thank You @MustafaJafar, its working great for LabsKarma, we can publish renders through it now!

I'm glad it worked

We saved as a new copy and added deadline and other features inside. I also matched the type of the HDA in lib.py. still its not working.

changes in lib.py requires restarting the Houdini app.

Every other studio/artist will have there own HDA. It would be great, if Ayon tools can be designed in a way that it can fit.

That's been one of me points since I started proposing a refactor of the Houdini OP/AYON integration. As it is it's very opinionated on the workflow the Houdini artist should work and Houdini is the most customized and flexible DCC there is, like @krishna8008 is saying every studio/TD will have their own ways of creating outputs (even their own farm integrations at times) and so AYON should be just a very thin layer on top of that in order to be able to integrate any output into the pipeline. If someone wants to work in a very simple blackbox workflow what it is right now works good... but that's not true for most and that's why I ended up creating a different integration.

tbh, sometimes I imagine generic creator as fabia mentioned in his demo and as the following mockup screenshot. where you specify the product type and list of files to include but this leads to losing the validators and other plugins.

This is not necessarily true. We should be able to run any validations without the current creator workflow. In fact, we should be able to implicitly create creators on vanilla nodes and close the bridge with those workflows

Hey! @BigRoy  @MustafaJafar @fabiaserra :
Like @fabiaserra and I are saying about the custom HDAs for any studio and individuals,. I was just wondering, instead hardcoding the type of the node in houdini, this
elif node_type == "labs::karma::2.0":
return node.parm("file")
Is there any better way to get the code working for custom node types, Hardcoding here in git will only work for Labs nodes; a dynamic change of script is probably for the custom HDAs support, or choosing any other method other than type?! So that the code works universally for everyone. A proper documentation will help the creators to guide them on how to do it in the right way, which is IMO a good way to go. I'm excited to see what @BigRoy was working on. Thanks Guys!

Thanks Krishna, the idea that I have is to try and expose some of these things to studio/project settings. Likely I will have that prototyped next week.

[ ] Make sure this API approach remains a goal of the PR.

Will context.publish publish using the publisher like here or the pyblish api like here

omg 😅

Will context.publish publish using the publisher like here or the pyblish api like here ?

This point actually relates more now to the separated PR #691 - but I'd say if we can go through CreateContext as that exposes more of an API by the way its designed and we have more control over designing it than the pyblish API.

Will it make use of context.publish ?
I imagine it as context.publish(my_instance)

Not necessarily - this PR now focuses mostly on the 'generalization' of ROP nodes in Houdini to detect them as publishable. CreateContext.publish() isn't currently an available API function - however, we could create a lib.publish(create_context) method for the time being - since functionally there isn't much to that but in essence that is 99% already what self-publish code is doing. It's just exposing that as a proposal API method then.

onCreate event is cool it made me wonder, Should the generic creator be visible in the Creator UI ?
Also, The default $OS is not favored by the creator UI.

omg 😅 is this expected/intended ?

Nope - pushed a hotfix for now. Had it on my radar and wanted to do it "the right way" but went on to other stuff.

Should the generic creator be visible in the Creator UI ?

I've had my doubts about that - what do you think?

Also, The default $OS is not favored by the creator UI.

Ah yes - that doesn't seem great. I'm starting to hate the publisher UI ;) Anyway, hacked around it with https://github.com/ynput/ayon-core/pull/542/commits/939ee37f69825eb3a563bf9ca3d8c7438757653b

For information. These don't match.

For information. These don't match.

Publisher attributs specific to the instance instead of the creator
Perfect - thanks. This will be hard to fix with the design of the publisher's logic unfortunately. A single Creator can't define its properties per instance, but only for the creator. That, by design, then limits how we can generalize these for a single Creator unfortunately since it can only have the state of one. Where in this case for render instances we should be displaying different properties.

There are some other major issues that I was hoping to write up - because @dee-ynput also mentioned in essence that's also a goal of this, basically figuring out what the needs are for an API and the UI to follow suit.
Other things to keep in mind are:
Single output path versus a ROP that may generate any amount of "linked files"

The generalized logic now where "Collect Frames" detects the output files from a ROP node based on an output parameter is a nice generalization but also fails for a lot of special cases. For example:

.gltf file path may be set to $HIP/my.gltf but actually exports more files that should also be integrated (and actually have their filenames preserved on the publish as well since the gltf links relatively to the other files!)
a USD rop may export more than one .usd file in any structure necessary. It may also save sublayers, etc. (which also would have to be relinked to any published structure or has the same issue as gltf publishes that the filepaths inside the files have a hard link)
a USD rop may spawn other products like asset USDs, layer USD files which could each also need relinking. This is basically somewhat what I had to face in PR #295 - which made it solvable by focusing purely on the USD aspect of things without thinking about generalization of logic beyond that point to e.g. gltf or other file formats. (and also relying specifically on the USD API, which of course does not translate to other file formats)

This may be somewhat resolved by exposing attributes to the user where they can configure additional files to ingest, but for complex cases the only real sensible way to do this - is to do it programmatically, because we don't want to continuously have artists need to manually set tons of files a particular ROP may output (if those are dynamic) which again may make it hard to abstract/generalize.
Although for a particular ROP node we could have a get_expected_outputs implementation that does that for GLTF, for USD, etc. but still it'd be specific to the ROPs; what then about HDAs? and where do we maintain those custom implementations?

A lot of this boils down to finding what is the easiest open-accessible, easy to code yet also easy to maintain API structure we can design for creators and publishing - where we hopefully can avoid as much as possible hardcoding 'specificness' in design. I had the same discussion with @fabiaserra recently where he said "we should rely less on Creators for publishing in Houdini" but then at the same time his codebase has 'wrappers' for Houdini node types that in a way are trying to define what something should publish as, what the default parm values are, etc. In essence what he's done is also creating an API for what we need to prepare something for publishing, basically designing his own "Creator" API.
In his implementation it could work to e.g. have (oversimplified pseudocode):
class UsdRopNode(ayon_houdini.NodeWrapper):
    def on_created(self, node):
        self.make_publishable(node)
        self.set_parm_defaults(node)

    def get_expected_files(self):
        return [file1, file2]

    def set_parm_defaults(self, node):
        # update the parm defaults for this node
        pass

We're just shifting where we're putting the logic. However, being able to access some of these bits outside of pyblish where we don't need "collect" and validations, etc. does make it trivial to abstract other bits.
Yet, as soon as that wrapper also need to expose what settings are user configurable, etc. we're basically expanding the Creator class design to be somewhat what they are now yet also include what @fabiaserra does. Meaning we actually have less generalizing of Creator logic but start getting more specific per node in Houdini which I feel is funnily enough the exact opposite Fabia has been arguing for a lot, and me as well.
TL;DR - finding the simplest API that can do all we want is hard, but should be the goal

I was trying submitting to farm and it may need additional work because:

"generic" is not added to the families of some deadline and houdini plugins. or Should we include the original product type e.g. mantra_rop as well as render ?
The logic of deadline plugins depends on product type.

TL;DR - finding the simplest API that can do all we want is hard, but should be the goal

Thanks for the explanation.

I'll close this PR and reopen it on seprated ayon-houdini repo once the addon separation has finished.


update my name and  link

Related issue

What changes does this PR do?
Screenshots (If any)
Questions

ok


Error processing ImageSet configuration

Version
$ oc-mirror version
v0.1.0-alpha.3

What happened?
I was trying to mirror operators using the snippet below as the imageset config file.
Wh:q
en I use the same structure without the version added to the file I got the mirror to run until I ran out of storage on the device but once I added the version I started getting the error below.
Attach the relevant portions of your .openshift_bundle.log.
There was nothing in the log file but I got the error below on the command prompt
error: decode tmp-redhatgov.com/v1alpha1, Kind=ImageSetConfiguration: No Major.Minor.Patch elements found
What you expected to happen?
A bundle created with list of operators defined in the the imageset config file.
How to reproduce it (as minimally and precisely as possible)?

$ oc-mirror --config operator-imageset-channels-config.yaml file:///data/registry-oc-mirror

Anything else we need to know?
The reason the above file only contain operators is because our existing processes that have various ansible playbooks dedicated to mirroring a single type of content like operators, OCP images, binaries, helm charts .... The goal is to simplify the process by replacing multiple tasks with the single oc-mirror command to produce the same bundle and.  So we will be defining a file for each type of content being mirrored.
Also it would be good to have sample config files with a couple of operators listed with versions in the readme file so people can use that as a guide or example to follow when creating their own imageset config.
Building the binary and running the list sub commands worked great. Those sub commands are very straight forward and easier to use that the current approach in the OCP docs. Great job on providing those to make our lives easier.
The other observation I have is that if you provide a list of operators without version, the mirror command does not seem to honor the list or if it does, it tries to pull all available images for each of the operators in the list. I was not able to finish the process because of no space left on the device error (after 265GB of content was mirrored into the workspace directory), I ran into while using a subset of the operators from the file above. I was expecting to get the latest version of the operand images like Arvin A. 's code does when a list of operators is provided.
We will need a way to specify that we only need the latest version without having to figure out the version and then add it to the imageset config file.
I also saw some other errors related to images not found (e.g. servicemeshoperator.v2.0.4 had missing images and was causing the mirror to fail until I added --skip-missing option to ignore that).
References

Enter text here.

:wave: @cadjai. Thank you for your feedback! Could you please post the imageset config you are working with? Also, we do have a config reference under docs, but I totally understand how having an examples directory that has configs with certain use cases would be helpful. I can submit a PR to add more example configurations today.

operator-imageset-channels-config.yaml.txt
Loading the file as .txt because yaml does not seem to be supported

@cadjai This looks like an issue with our docs. We haven't done a good job of documenting the config (especially for operators) To demo redhat-operator-catalog:v4.7 / elasticsearch-operator from your config. For me to find the version of the operator, I would use the following workflow:
alex@penguin:~/dev/bundle$ ./bin/oc-mirror list operators --catalog=registry.redhat.io/redhat/redhat-operator-index:v4.7 --package=elasticsearch-operator
PACKAGE                 CHANNEL     HEAD
elasticsearch-operator  5.0         elasticsearch-operator.5.0.9-50
elasticsearch-operator  stable      elasticsearch-operator.5.2.2-21
elasticsearch-operator  stable-5.1  elasticsearch-operator.5.1.4-9
elasticsearch-operator  stable-5.2  elasticsearch-operator.5.2.3-31
alex@penguin:~/dev/bundle$ ./bin/oc-mirror list operators --catalog=registry.redhat.io/redhat/redhat-operator-index:v4.7 --package=elasticsearch-operator --channel=stable-5.2
VERSIONS
5.2.3-31

From what I can tell, your config conflates operator package channels and versions. A valid elasticsearch section from your config would be correct like this:
        - name: elasticsearch-operator
          versions:
            - '5.2.3-31'

Operator selection behavior will be getting an overhaul shortly, so some of the logic processing of your config will change soon. Hopefully that makes it easier to author these configs too.

The behavior pertaining to operators without version specification will benefit from the overhaul that I referenced above.

Hello Alex
Thank you for addressing some of my comments. I will give it a try again.
Also to clarify how I got the version I am using I ran something like the following commands
oc-mirror list operators --catalog=registry.redhat.io/redhat/redhat-operator-index:v4.7
oc-mirror list operators --catalog=registry.redhat.io/redhat/community-operator-index:v4.7

and that gave me a list of the operator with versions that I plugged in the imageset so that I didn't have to run the command one at a time but I will give that a try.
Thank you


Add parameter in New-SM365Connectors -NoEfSkipIP

used for IP Addresses which are in prvate networks (not spported my MS)

mgmt in internal repo


Add "forest" package

I'm interested in package <forest>

Official GitHub repository: https://github.com/xorz57/forest

Used build system: [header-only does not require build system]

This package has no dependencies

header-only does not require build system

In general it's better to use build system even for header-only libraries:

library should be tested
library may have dependencies even if it's header-only itself
you may want to install *Config.cmake files for integration into other projects

Adding CMake installation code to header-only library is a trivial effort:

https://github.com/ruslo/leathers/blob/4bd0f6c36468c479f6010c45ab4f5f919c11ed49/CMakeLists.txt#L91-L141


[AutoPR mariadb/resource-manager] Update MariaDB default API version to 2018-06-01

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/5302

(message created by the CI based on PR content)

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/3980