Boundary Loss Training ?

I want to use the boundary loss with my dataset but the conversion between masks to contours can be a pain, is there a simpler way to do it ?

Hi @MohamedAliRashad ,
We have a demo reimplementation at
https://github.com/JunMa11/SegWithDistMap/blob/8c3c7656f8259bff2d31509a881553d5cf5bb691/code/train_LA_BD.py#L121-L133
and
https://github.com/JunMa11/SegWithDistMap/blob/8c3c7656f8259bff2d31509a881553d5cf5bb691/code/train_LA_BD.py#L192-L203
Hope it could help you.
Best,
Jun

Thanks @JunMa11 but my problem is in the training (currently), is there a way to make the DistLoss take the same inputs like the CrossEntropyLoss from pytorch.
network_output=(Batch, Num_classes, H, W)
target_shape=(Batch, H, W)

Hi @MohamedAliRashad ,
You may need to use one-hot encode to labels or manually specify one dimension like this
https://github.com/JunMa11/SegWithDistMap/blob/8c3c7656f8259bff2d31509a881553d5cf5bb691/code/train_LA.py#L108-L110

@JunMa11 How this will enable me to use it with the Boundary Loss ?
And while we are on the subject, what compound loss would you suggest for an imbalanced, hight IoU requirement ?

Hi @MohamedAliRashad ,
To use boundary loss, please refer to this out-of-the-box code.

what compound loss would you suggest for an imbalanced
Based on my experience, using BD loss+ Dice loss is better than only Dice loss.
However, it should be noted that this is still an unsolved problem.

@JunMa11 I have some questions if you can help me.

what is the precomputed distance map is used for ?
Does the implementations of DC_and_BD_loss support normal images like it supports volumetric ones ?
if i want to combine the two losses is it better to just sum them or should i add weights ? and if i should add weights, how to determine them ?

Hi @MohamedAliRashad ,
Sorry for my late reply.

boundary loss relies on the distance map.
No.
According to the original paper, in the beginning, larger weights should be given to dice loss, and then gradually increase the weights of BD loss.
apply_nonlin means apply softmax to the network logits output.

Again, please refer to
https://github.com/JunMa11/SegWithDistMap/blob/8c3c7656f8259bff2d31509a881553d5cf5bb691/code/train_LA_BD.py#L192-L203
if you want to use boundary loss for your own tasks.
Best,
Jun

@JunMa11 I am sorry for the disturbance, I am just trying to figure out how to adjust the DC_and_BD_loss function for a 2D input.

from scipy.ndimage import distance_transform_edt as distance
from skimage import segmentation as skimage_seg
def compute_sdf(img_gt, out_shape):
    """
    compute the signed distance map of binary mask
    input: segmentation, shape = (batch_size, x, y, z)
    output: the Signed Distance Map (SDM)
    sdf(x) = 0; x in segmentation boundary
             -inf|x-y|; x in segmentation
             +inf|x-y|; x out of segmentation
    """

    img_gt = img_gt.astype(np.uint8)

    gt_sdf = np.zeros(out_shape)

    for b in range(out_shape[0]): # batch size
        for c in range(1, out_shape[1]):
            posmask = img_gt[b]
            negmask = 1-posmask
            posdis = distance(posmask)
            negdis = distance(negmask)
            boundary = skimage_seg.find_boundaries(posmask, mode='inner').astype(np.uint8)
            sdf = negdis - posdis
            sdf[boundary==1] = 0
            gt_sdf[b][c] = sdf

    return gt_sdf

def boundary_loss(outputs_soft, gt_sdf):
    """
    compute boundary loss for binary segmentation
    input: outputs_soft: softmax results,  shape=(b,2,x,y)
           gt_sdf: sdf of ground truth (can be original or normalized sdf); shape=(b,2,x,y)
    output: boundary_loss; sclar
    """
    pc = outputs_soft[:,1,...]
    dc = gt_sdf[:,1,...]
    multipled = torch.einsum('bxy, bxy->bxy', pc, dc)
    bd_loss = multipled.mean()

    return bd_loss

        with torch.no_grad():
            gt_sdf_npy = compute_sdf(label_batch.cpu().numpy(), outputs_soft.shape)
            gt_sdf = torch.from_numpy(gt_sdf_npy).float().cuda(outputs_soft.device.index)

        loss_boundary = boundary_loss(outputs_soft, gt_sdf)
        loss = alpha*(loss_dice) + (1 - alpha) * loss_boundary # alpha = 1.0
        alpha -= 0.01
       if alpha <= 0.01:
           alpha = 0.01

@JunMa11 yeah and also make it multi-class :sweat_smile:

@JunMa11
How would I use the Boundary loss in a multi class scenario?
How would I compute the sdf map for multi class?
Thank you!


Crash error: logger with name 'wasi_logging_stdout' already exists

The application crashes when a second model is loaded, showing the following error:
libc++abi: terminating due to uncaught exception of type spdlog::spdlog_ex: logger with name 'wasi_logging_stdout' already exists

It happens in the following flows:

Choosing a new model in the main dropdown in the Chat screen
Using the "Start Chat" buttons in My Models view
Switching to another conversation (in Chat screen) where a different model was the last one used.

In all cases, the backend seems to struggle to load a new model.

@hydai https://github.com/moxin-org/moxin/blob/fb3d19c0307ff11d71f55f99665d13e728f34614/moxin-backend/src/backend_impls/api_server.rs#L100-L107

@jmbejar  Try this simple way to fix the problem , delete the WasiNNLogging plugin from your plugins folder

@L-jasmine After removing ~/.wasmedge/plugin/libwasmedgePluginWasiLogging.dylib the issue didn't happen again, so the solution to this issue is related with this plugin. What should be done to avoid this issue in other users?

Can we provide a version check to make sure the FE and BE stays consistent - and give a message to user when things are out of alignment and need further steps (such as re-installing something)


[Feature] Added batch_lock attribute in EnvBase

Description
This PR solves the problem to distinguish if batch size of an env is flexible or not
Motivation and Context
The motivation is that stateful envs have different behavious than stateless envs with respect to batch_size. Where as stateful envs have a hard constraint on env.batch_size == tensordict.batch_size it's not the case of stateless envs. Therefore we add batch_locked attribute that allow or not batch flexibility in the env
Types of changes
What types of changes does your code introduce? Remove all that do not apply:

[x] New feature (non-breaking change which adds core functionality)

Checklist
Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

[x] I have read the CONTRIBUTION guide (required)
[x] I have updated the tests accordingly (required for a bug fix or a new feature).

I've added tests for gym. I'll add the same test for MB_env in the MBEnv PR #333

I think tests are failing because you have not merged main into your branch recently but i might be wrong

Why would we make TransformedEnv being able to change the batch_locked. At the end of the day, base_env will still have that constraint

@nicolas-dufour have a look at my changes, if you're ok with them we can merge


Why is the API marking Chinese cdns as unsafe while online/browser lookup aren't ?

Hi guys,
Thanks for this amazing tool.
As previously mentioned in this issue, many folks are seeing discrepancies between the client and safe browsing-enabled browsers.
I specifically observed that a huge of amount of cdns/apis of Chinese services are marked as unsafe by the api client. Examples include  "hlsa-akm.douyucdn.cn", "gslb.miaopai.com/", and "switch.pcfg.cache.wpscdn.cn". These sites are marked as unsafe by the API. However, the transparency report says it is SAFE.
A sample query result from the API.
{
  "matches": [
    {
      "threatType": "UNWANTED_SOFTWARE",
      "platformType": "ANDROID",
      "threat": {
        "url": "gslb.miaopai.com/"
      },
      "cacheDuration": "300s",
      "threatEntryType": "URL"
    },
    {
      "threatType": "UNWANTED_SOFTWARE",
      "platformType": "ANDROID",
      "threat": {
        "url": "switch.pcfg.cache.wpscdn.cn"
      },
      "cacheDuration": "300s",
      "threatEntryType": "URL"
    }
  ]
}

Any ideas on what's up? I am also curious why these sites are considered "UNWANTED_SOFTWARE" in "ANDROID" only. Thanks

For future reference, there's a bug in Safe Browsing's diagnose tool, and they are working on fixing it


texstudio: 2.12.20 -> 2.12.22

Also add myself as maintainer.

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Is https://github.com/NixOS/nixpkgs/issues/79034 also solved with this?


fix: Improve PES packet handling

Check PES packet's length for non video (pes_packet_lenght != 0) PES.

Merged, thanks!


ActiveField with $options['tag'=>false] outputting empty (not valid) tag.

What steps will reproduce the problem?
$form = ActiveForm::begin([
    'id'          => 'create',
    'fieldConfig' => [
        'template'=>'{label}
{input}{hint}{error}',
        'options' => [
            'tag'=>false,
        ],
    ],
]); ?>

<?= $form->field($model, 'name')->textInput(['maxlength' => true]); ?>

What is the expected result?
Name
<input type="text" id="object-name" class="form-control" name="Object[name]" maxlength="150">

What do you get instead?
< class="field-object-name required">
Name
<input type="text" id="object-name" class="form-control" name="Object[name]" maxlength="150">

</>

Additional info

Q
A

Yii version
2.0.?

You can only change the tag, not omit it.
Do you have a use case for trying to remove it?

Have you tried latest master code?

@Alex-Code documentation say that when the value is false, the container tag will not be rendered at all.
http://www.yiiframework.com/doc-2.0/yii-widgets-activefield.html#$options-detail
@samdark No, i'm using 2.0.6

Well, upgrade.

The issue was fixed. Either it's in 2.0.10 or in 2.0.11 which is to be released.

Should be #11949 which is part of 2.0.10.

ok, thanks. Sorry for disturbing.


Toggling tag when only one is available doesn't take effect until focus changes

Right-clicking the only tag in view to toggle it out of view doesn't take effect until the cursor hovers over a client. This might have something to do with the toggleview command not rearranging the view.

This was caused by a conflict between the emptyview patch and pertag patch. The pertag patch overrode the emptyview patch causing the focus and arrange functions to not be called if the toggleview function results in no tags selected. The reason this was working fine with key bindings is the buttonpress function in dwm calls focus and arrange regardless.

For future reference, see https://github.com/mihirlad55/dwm/commit/e2650a679bed382ef921234bc67aac91c68c87f6

That was actually not the main issue. The main issue was the XMoveWindow function call that is supposed to take place in showhide would not be flushed to the XServer. Normally this wouldn't be a problem because an XEvent would being received would trigger a flush or XSync would be called from restack which would manually flush the events. However, if no client is selected, the restack function returns before XSync is called. This would result in the client still being visible since the XMoveEvent was not being flushed to the server until another XEvent took place that causes the queue to be flushed.
There are several solutions to this problem:

XSync after all IPC commands - This makes the IPC patch responsible for this bug
XSync even in restack even if no clients are selected - This makes the emptyview patch responsible for this bug
Receive motionnotify events from external bars managed using the anybar patch - this isn't a direct solution, but it should solve the problem

I believe, this is mainly an issue of an unforeseen edge case. Though, an empty view is not something dwm was necessarily designed to allow by default. I think this would be best classified as an unforeseen edge case of the emptyview patch, since the restack function would otherwise never need to XSync if no clients are selected unless there was an empty view.

See https://github.com/mihirlad55/dwm/commit/ea047a44e6dde9fc7712b45fb7f6d9d74d82f976


support globs & lists in osfinger key

We're starting to find some duplication in data between distributions (usually within the same os family). It would be nice to support globbing and lists in the osfinger setting:
pkg:
  blacklist:
    telnet-server:
      data:
        ['CentOS-6', 'CentOS Linux-7']:
          - 'telnet-server': 'CIS-XX'
      description: 'Telnet server is teh bad'

Implemented, though you have to use comma-separated lists, not pythonic lists. (Lists cannot be keys in dictionaries)


 and other meta tags are being read aloud as if part of speech

Must be setting these tags incorrectly, as on the test portal, it is treated as part of the SSML and interpreted, but IRL when speaking to the Echo, it reads out the tag as if it was regular text

From the docs, we needed to be using SsmlOutputSpeech class
https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/speech-synthesis-markup-language-ssml-reference


add Source CrystalComics "pt-br"

Closes #1692
Checklist:

[ ] Updated extVersionCode value in build.gradle for individual extensions
[x] Updated overrideVersionCode or baseVersionCode as needed for all multisrc extensions
[x] Referenced all related issues in the PR body (e.g. "Closes #xyz")
[ ] Added the isNsfw = true flag in build.gradle when appropriate
[ ] Have not changed source names
[ ] Have explicitly kept the id if a source's name or language were changed
[x] Have tested the modifications by compiling and running the extension through Android Studio
[x] Have removed web_hi_res_512.png when adding a new extension

Please actually test your changes


create an install type that is like the docker-machine one

ie, lays down a RANCHER_STATE partition, copies over any ssh keys or config, copies over any images,
and add an upgrade boot sdcard / usb key / download iso.
that way, my old b2d server can continue to boot and run from a usb key, with /var/lib/docker and cfg being on the disk. - but with the added bonus that it can upgrade itself.
this is kinda like a poor-network version of a good PXE setup - so in the docs highlight both.

it may be un-necessary to code anything more than a set of default hints showing the right cloud-config to make it happen..


editor prompt: only call issubclass when the "validate" argument is a class

For some reasons I don't really understand sometimes this causes an exception, sometimes it does not. Anyway, issubclass only access class objects as first argument so when the "validate" field is just defined as a function an error is raised. Using inspect to check that the passed value is actually a class should fix the issue (this was already employed in prompts/input.py).

@piax93 Thanks for the PR


Adding Milk Mod Economy

Hello!  Wanted to start by saying I've been enjoying this mod but was disapointed to find out it doesn't support Milk Mod Economy.  I tried to spend time today to see if I could get it working myself but just ran into issue after issue trying to get things setup.  CMake just refused to work and other things just failed as well.  I did end up managing to figure out what events should be used for the listeners and prototype up the below taking advantage of the existing devious devices setup.  Anyway, would love to see this get added but would very much understand if it is not something you want to put the effort in.  I wish I could be doing this as a PR but its just not working out that way (unless you would like me to go ahead and try to do that without being able to test it.)
Thank you!
;               __  ____ ____      __  ___          __   ______
;              /  |/  (_) / /__   /  |/  /___  ____/ /  / ____/________  ____  ____  ____ ___  __  __
;             / /|_/ / / / //_/  / /|_/ / __ \/ __  /  / __/ / ___/ __ \/ __ \/ __ \/ __ `__ \/ / / /
;            / /  / / / / ,<    / /  / / /_/ / /_/ /  / /___/ /__/ /_/ / / / / /_/ / / / / / / /_/ /
;           /_/  /_/_/_/_/|_|  /_/  /_/\____/\__,_/  /_____/\___/\____/_/ /_/\____/_/ /_/ /_/\__, /
;                                                                                /____/

Function InitMilkModHandlers()
        RegisterForModEvent("MilkQuest.StartMilkingMachine", "OnMilkModVibrateEffectStart")
	RegisterForModEvent("MilkQuest.StopMilkingMachine",  "OnVibrateEffectStop")

	RegisterForModEvent("MilkQuest.FeedingStage", "OnMilkModVibrateEffectStart")
	RegisterForModEvent("MilkQuest.MilkingStage",  "OnMilkModVibrateEffectStart")
	RegisterForModEvent("MilkQuest.FuckMachineStage", "OnMilkModVibrateEffectStart")
EndFunction

Bool _MilkMod_Vibrate = false
Bool Property MilkMod_Vibrate Hidden
    Function Set(Bool enable)
        _MilkMod_Vibrate = enable
    EndFunction
    Bool Function Get()
        return _MilkMods_Vibrate
    EndFunction
EndProperty

Event OnMilkModVibrateEffectStart(String eventName, String actorName, Float vibrationStrength, Form sender)
    If !_DeviousDevices_Vibrate || ZadLib == None
        return
    EndIf

    If PlayerRef.GetLeveledActorBase().GetName() != actorName
        return ; Not the player
    EndIf

    String[] events = new String[3]
    Float numVibratorsMult = 0
    If eventName == "MilkQuest.FeedingStage"
        numVibratorsMult = 0.3
        events[1] = DeviousDevices_Vibrate_Event_Anal
    EndIf
    If eventName == "MilkQuest.MilkingStage"
        numVibratorsMult = 0.55
        events[1] = DeviousDevices_Vibrate_Event_Anal
        events[2] = DeviousDevices_Vibrate_Event_Nipple
    EndIf
    If eventName == "MilkQuest.FuckMachineStage"
        numVibratorsMult = 0.95
        events[1] = DeviousDevices_Vibrate_Event_Anal
        events[0] = DeviousDevices_Vibrate_Event_Vaginal
    EndIf
    If eventName == "MilkQuest.StartMilkingMachine"
        numVibratorsMult = 1.25
        events[0] = DeviousDevices_Vibrate_Event_Vaginal
        events[1] = DeviousDevices_Vibrate_Event_Anal
        events[2] = DeviousDevices_Vibrate_Event_Nipple
    EndIf

    Int strength = Math.Floor((vibrationStrength / numVibratorsMult) * 20)
    TDevices.StopHandle(_DeviousDevicesVibrateHandle)
    _DeviousDevicesVibrateHandle = StartVibration(DeviousDevices_Vibrate_DeviceSelector, -1, DeviousDevices_Vibrate_Pattern, DeviousDevices_Vibrate_Funscript, strength, events)
EndEvent

Hey, very cool Idea, I didn't think of that mod.
A few things:

There is no need to use CMake etc to extend this mod you mainly need to do that only for the native library, which I assume you don't want to change / rebuild.
You can build your own esp, make it depend on Telekinesis.esp, fetch the main quest (like I do here with other mods) Tele_Devices.psc, and just start using the methods exposed by that method. But if you do, please tell me so that I stop changing the API (lol).

There are a few problems with your code and I won't be able to merge that:

StartVibration is an internal helper method I utilize to access some common MCM settings for vibrations. You are using the Devious Devices Settings, but you would need to create your own set of settings in the MCM for Milk Mod Economy, or you use TDevices.Vibrate directly.

You overwrite the _DeviousDevicesVibrateHandle that is used to track currently running DD vibration tasks, which could result in them not stopping cause it lost the handle.

I would be interested in adding that at some point, and your code is helpful to see how it could be done.

I don't know when / how I would include it (too much to do)

Right now I'm also working on a separate project that would allow you to define such things by json configuration, i.e. mapping in-game events to specific device actions.

Hello! Wanted to start by saying I've been enjoying this mod but was disapointed to find out it doesn't support Milk Mod Economy. I tried to spend time today to see if I could get it working myself but just ran into issue after issue trying to get things setup. CMake just refused to work and other things just failed as well. I did end up managing to figure out what events should be used for the listeners and prototype up the below taking advantage of the existing devious devices setup. Anyway, would love to see this get added but would very much understand if it is not something you want to put the effort in. I wish I could be doing this as a PR but its just not working out that way (unless you would like me to go ahead and try to do that without being able to test it.)

Can you elaborate on what you have to do to get those events to trigger in Milk Mod Economy?
I tried using the milk machine with my PC, nothing happens.
I tried to drink the lactaid and become a milk maid, and nothing happens again.

I actually got it fully working myself and I've been meaning to PR it back
in but have not gotten to that yet.  I think I'll try to do that today.
The events should trigger when using the milk machine.  The version I have
is working.
On Thu, Mar 28, 2024 at 4:57 PM garryrot @.***> wrote:

Hello! Wanted to start by saying I've been enjoying this mod but was
disapointed to find out it doesn't support Milk Mod Economy. I tried to
spend time today to see if I could get it working myself but just ran into
issue after issue trying to get things setup. CMake just refused to work
and other things just failed as well. I did end up managing to figure out
what events should be used for the listeners and prototype up the below
taking advantage of the existing devious devices setup. Anyway, would love
to see this get added but would very much understand if it is not something
you want to put the effort in. I wish I could be doing this as a PR but its
just not working out that way (unless you would like me to go ahead and try
to do that without being able to test it.)
Can you elaborate on what you have to do to get those events to trigger in
Milk Mod Economy?
I tried using the milk machine with my PC, nothing happens.
I tried to drink the lactaid and become a milk maid, and nothing happens
again.
—
Reply to this email directly, view it on GitHub
https://github.com/garryrot/telekinesis/issues/2#issuecomment-2026338411,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEW6ZWSYZXTZM35UXDJ5PSLY2SN5NAVCNFSM6AAAAABEE5WLXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRWGMZTQNBRGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

I actually got it fully working myself and I've been meaning to PR it back in but have not gotten to that yet. I think I'll try to do that today. The events should trigger when using the milk machine. The version I have is working.
…

Do you run the SE version from LL? https://www.loverslab.com/files/file/6103-milk-mod-economy-se/
Milk-Mod-Economy-Scripts-20220522.7z

That looks right.  I just got the PR up if you want to take a look at how I implemented it.  I tried to copy your existing setup for things.

That looks right. I just got the PR up if you want to take a look at how I implemented it. I tried to copy your existing setup for things.

Yeah I figured the event issue, it was skill issue (on my part).
I'm currently working on a project to hook to events with a generic json configuration ( see here if interested ), but unfortunately that seems to just not work for events that are non-standard parametrized.
I want to have a more generalized solution, to save everyone all of the trouble you had to go through, but that also ate my time and kept me from implementing new features.

Solved by merging https://github.com/garryrot/telekinesis/pull/4


window is not defined

When I am using this library with Next.js is returns the error.
I think it's a problem with SSR.

Doing a dynamic import and setting { ssr: false } still doesn't even resolve the issue. A blocker to using this lib.

The problem has been fixed.


[4.x] - Tracing Test Fail

Intermittent:
io.helidon.tracing.tests.it1.OpentraceableClientE2ETest.e2e  Time elapsed: 0.358 s  <<< FAILURE!
java.lang.AssertionError:
Expected: a collection with size <0>
     but: collection size was <1>

Failing (quite consistently) on local machine.
Disabling test.

Test is now refactored, I will create a PR soon.

This should now be fixed. There was an interference from the other unit test, as it would create a span, activate it and then never closes the scope. I have refactored the filter tests to run on a different thread, so the thread local does not impact any other code.


Can propose same agreement several times, even if already agreed to (is this intentional?)

pretty sure it is intentional since we have our own logic to include the (again) label into the button


firewall log widget: allow multiple choice

Hi!
replaces https://github.com/opnsense/core/pull/4817 and possibly closes https://github.com/opnsense/core/issues/4815

no legacy_config_get_interfaces
multiple choice of interfaces
no toLowerCase()
not reading DOM attrs for each log entry
starts on "WidgetsReady" for Firefox consistency on page refresh (bootstrap not applied on 'load' after F5. so FF generates second <button>)
widget works on driver-names and is indifferent to upper\lowercases from the API
thanks!

@AdSchellevis Hi! sorry, I prepared for a pr for https://github.com/opnsense/core/issues/4836 (not so much code realy). may I combine changes from this pr with the new one. or should I make them separate and then resolve the conflict in one of them (if\when someone will be merged)?

If it touches the same file I think it’s ok

On 25. Mar 2021, at 07:44, kulikov-a @.***> wrote:

@AdSchellevis Hi! sorry, I prepared for a pr for #4836 (not so much code realy). may I combine changes from this pr with the new one. or should I make them separate and then resolve the conflict in one of them (if\when someone will be merged)?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

@fichtner yep. Thanks! preparing a combined pr with explanations

seems to be ready for review. decided to add commits to this pr to make it easier to distinguish changes for different FRs on review

@kulikov-a looks really nice, personally I would prefer to keep the <select id='filterlogentriesinterfaces' .. as it was to prevent cluttering php and javascript, but I'll let it go this time since technically it's not wrong. I did wrap and improve your parameter parsing in the live view a bit (https://github.com/opnsense/core/commit/352d5de372cb0805ad631de67b0a238324472134), so we can easily reuse this later to store selections (in localstorage?), I think that was also a ticket somewhere which probably makes sense and shouldn't be to hard to add.
Thanks a lot for  your efforts!

@AdSchellevis Wow thanks!!
yes, my parsing was very primitive. thanks for making it really useful )

@kulikov-a if you're looking for a new challenge, you could try to add a "save local selection" option, so you can easily switch between presets (https://github.com/opnsense/core/issues/4859). No pressure, only if you have the spirit for it.

Very nice, thank you @kulikov-a

@AdSchellevis @fichtner thanks!! :blush:
yes, it would be interesting to try to make stored filter templates though I'm afraid you will have a lot of work fixing my code)) and I might be a little slow until early May (moving the office. a lot of stupid time-consuming work starts). but if there is no strict deadline, I would love to.

@kulikov-a let's try it out and see how that goes, I've assigned you to the ticket, there's really no rush at all. Don't mind reviewing some of your code, it's quite decent and has lead to quite some nice improvements over the last months. Thanks!


BundleCache - Filter CacheDependency files

This is a a small bug fix for BundleCache, to filter files that is monitored for changes by a CacheDependency, so that only physical files that can be monitored is included in the file list. My use case was that a script bundle included a "non-physical" file exposed by a virtual path provider. It is otherwise fully supported by Squishit, but it threw an exception when the CacheDependency was created.
This change is backward compatible and shouldn't negatively affect any existing code. All tests pass, of course. Please tell me if I need to improve something!

This looks reasonable - good find.  Sorry about the delay.


Hide y-axis?

Hi there,
Is it possible to hide the y-axis in 1.0.2? I realize I can use scaleShowLabels to hide the labels, but I'd like to not draw a y-axis at all.
I've looked at the open issues and checked SO as well, but haven't found anything... guessing if it is doable it'd be via extending the Line chart, but I'm not sure where to start looking to figure that out.
Anyone have any suggestions?

Managed to pull it off with an edit to this SO answer to a similar question, but still interested in knowing if there's a better way to do this.

@oladon in v1 this requires a new chart type as the answer you found shows. Thanks for coming back to post the answer!
For anyone who searches across this, in v2 you can set display: false in the axis config and it will not draw.

@etimberg Thanks for the response. Even using a new chart type, it seems like there'd be a better way to do it than effectively "erasing" the axis after it's drawn... is that really the recommended way?
Maybe I should switch to v2...

@oladon The alternate would be to change the draw function for the axis to only draw the x stuff. In v1 is harder because the scale object represents both the x & y dimensions.
Even better might be to hide these lines behind an option: https://github.com/nnnick/Chart.js/blob/master/src/Chart.Core.js#L1733-L1780

@etimberg Guess I'll stick with the eraser for now, but thanks again for your helpful responses.


Docker Container-based geoprocessing

Support running of more complex and longer-duration geoprocessing tasks using Fargate.

Closing until first use case come along, we're aware we'd like to do this, possibly using the new Lambda container support.


Isn't this another case of Wix's DIY syndrome?

Honest question – doesn't Docker Compose, Docker Swarm, Docker Machine, Docker Toolbox, Crane, Shipyard, or Helios – among sooooo many others – already solve this problem? Isn't this another case of Wix's Do It Yourself syndrome?

Well, yeah.
We could've researched the subject, learn the relevant tools and make some decisions - maybe also regarding the production deployment and orchestration tools.
However, we needed something now.
The idea behind Sokoban is a dead-simple orchestration tool directed at E2E tests. It's open-source mainly so that us Node.js people can reuse it.
Eventually I hope we come to use a proper orchestration tool for our production environment - at which points the E2Es should definitely exercise this orchestrator.


Mavenize Play! Framework v.1.4.4

Version 1.4.4 was released on 2017.01.20.

Version 1.4.4 deployed to Maven central repository.


Change Rails.inject to look for the railties gem, not rails.

rails is a meta gem that can be skipped in a Rails app if we don't want to install gems like actionmailer, activerecord or activejob, while the railties gem is always present in a Rails app.
To ensure that Raven is properly loaded when the rails gem is not present we look for the railties gem, not rails in the Gem.loaded_specs Hash.

:+1: Great catch.


Drop @expectedException in favour of self::expectException

There are a bunch of @expectedException* annotations left in the unit tests which should be replaced by the corresponding self::expectException*() method calls due to being "a new best practice" as well as for consistency reasons.

@SenseException mind helping with this?

I can do that.


Small ADF updates

Add case nicknames
Add check in climo years for years that have leading zeros

Update from previous PR

Add obs file and variable name to plot titles
Add log-p plots for zonal maps
Update AMWG config file to match updates in baseline example config file
Add check for number of levels in difference plots

@nusbaume Looks like I didn't get them pushed. They have been committed now.


Error interacting with object store: Generic S3 error: Missing region

I've create a very simple, small Delta Table on my minio instance. Whatever I do to query it, I get the following error:
IO Error: Hit DeltaKernel FFI error (from: get_default_client in DeltaScanScanBind):
Hit error: 5 (GenericError) with message (Generic delta kernel error: Error interacting with object store:
Generic S3 error: Missing region)

The table was created from the following parquet file (which is located in the same minio instance and I can query fine with duckdb):
D SELECT * FROM 's3://vilrokx/prices.parquet';
┌─────────┬─────────────────────┬───────┐
│ ticker  │        when         │ price │
│ varchar │      timestamp      │ int64 │
├─────────┼─────────────────────┼───────┤
│ APPL    │ 2001-01-01 00:00:00 │     1 │
│ APPL    │ 2001-01-01 00:01:00 │     2 │
│ APPL    │ 2001-01-01 00:02:00 │     3 │
│ MSFT    │ 2001-01-01 00:00:00 │     1 │
│ MSFT    │ 2001-01-01 00:01:00 │     2 │
│ MSFT    │ 2001-01-01 00:02:00 │     3 │
│ GOOG    │ 2001-01-01 00:00:00 │     1 │
│ GOOG    │ 2001-01-01 00:01:00 │     2 │
│ GOOG    │ 2001-01-01 00:02:00 │     3 │
└─────────┴─────────────────────┴───────┘

Tried:
D SET s3_endpoint='<IP_ADDRESS>:9000';
D SET s3_use_ssl=false;
D SET s3_url_style='path';
D SET s3_region='us-east-1';
D SELECT * FROM delta_scan('s3://delta-lake/prices_table');
IO Error: Hit DeltaKernel FFI error (from: get_default_client in DeltaScanScanBind): Hit error: 5 (GenericError) with message (Generic delta kernel error: Error interacting with object store: Generic S3 error: Missing region)
D SELECT ticker FROM delta_scan('s3://delta-lake/prices_table');
IO Error: Hit DeltaKernel FFI error (from: get_default_client in DeltaScanScanBind): Hit error: 5 (GenericError) with message (Generic delta kernel error: Error interacting with object store: Generic S3 error: Missing region)

I am setting the region, as you can see, even though this should not be needed for a local Minio instance I would assume.
When I do the same with a Delta table on my local file system I get a different error:
D SELECT * FROM delta_scan('file:///prices_table') limit 1;
IO Error: Hit DeltaKernel FFI error (from: snapshot in DeltaScanScanBind): Hit error: 15 (MissingVersionError) with message (No table version found.)

FYI, this works perfectly fine on the same Delta Table:
from deltalake import DeltaTable

table_path = "prices_table"

df = DeltaTable(table_path).to_pandas()

print(df)

try this: CREATE SECRET (TYPE S3, provider credential_chain);

Yea the delta extension only works with DuckDB's secrets. So if you have a non-default region you will need to create an S3 secret with the correct region. @marsupialtail's solution will work if you have an s3 config in a standard location with your region, otherwise you can use:
CREATE SECRET (
    TYPE S3,
    REGION 'my-region'
);
FROM delta_scan('....');

What about all the other settings?
D SET s3_endpoint='<IP_ADDRESS>:9000';
D SET s3_use_ssl=false;
D SET s3_url_style='path';

When I set the secret, as per @samansmink's comment, I get:
IO Error: Hit DeltaKernel FFI error (from: snapshot in DeltaScanScanBind): Hit error: 8 (ObjectStoreError) with message (Error interacting with object store: Generic S3 error: Error after 10 retries in 2.227643s, max_retries:10, retry_timeout:180s, source:error sending request for url (https://s3.my-region.amazonaws.com/delta-lake/prices_table/_delta_log/_last_checkpoint): error trying to connect: dns error: failed to lookup address information: nodename nor servname provided, or not known)

which seems to suggest that it is not picking up the s3_endpoint I SET.

Yea no not yet, I'm only passing region and credentials atm, the rest should be easy to add (see src)
I will try to get to this soonish

Yea no not yet, I'm only passing region and credentials atm, the rest should be easy to add (see src)

Yeah that would be needed if you want to support e.g. minio

Came here to say that endpoints are being ignored, hence Delta on Minio is not supported yet.
INSTALL httpfs;
LOAD httpfs;

INSTALL delta;
LOAD delta;

CREATE SECRET secret_minio (
    TYPE S3,
    ENDPOINT 'localhost:9000',
    URL_STYLE 'path',
    USE_SSL false,
    KEY_ID 'minioadmin',
    SECRET 'minioadmin'
);

SELECT * FROM read_parquet("s3://test/submissions.pq");  # Works
SELECT * FROM delta_scan('s3://reddit-submissions/submissions-raw');  # Doesn't even hit http://localhost:9000

I'm seeing the same issue (DuckDB v1.0.0 1f98600c2c)
Set the MinIO config as a secret:
🟡◗ CREATE SECRET secret1 (
      TYPE S3,
      KEY_ID 'admin',
      SECRET 'password',
      REGION 'us-east-1',
      ENDPOINT 'minio:9000',
      URL_STYLE 'path',
      USE_SSL 'false'

  );

Confirm that DuckDB can read from MinIO:
🟡◗ select * from 's3://warehouse/part-85ca931a-7502-4736-964b-6b6127d7547a-0.snappy.parquet'
UNION ALL
select * from 's3://warehouse/part-c575067f-bb5c-471a-a344-45efe7dead2b-0.snappy.parquet';
┌─────────┬───────┐
│   c1    │  c2   │
│ varchar │ int32 │
├─────────┼───────┤
│ a       │    42 │
│ a       │    42 │
└─────────┴───────┘
Run Time (s): real 0.031 user 0.009555 sys 0.002291

Try to read the Delta table:
🟡◗ SELECT * FROM delta_scan('s3://warehouse/');
IO Error: Hit DeltaKernel FFI error (from: While trying to read from delta table: 's3://warehouse/'): Hit error: 8 (ObjectStoreError) with message (Error interacting with object store: Generic S3 error: Received redirect without LOCATION, this normally indicates an incorrectly configured region)

Confirm that the Delta table is valid, by copying it to local filesystem and then reading it locally:
🟡◗ SELECT * FROM delta_scan('/Users/rmoff/warehouse/');
┌─────────┬───────┐
│   c1    │  c2   │
│ varchar │ int32 │
├─────────┼───────┤
│ a       │    42 │
│ a       │    42 │
└─────────┴───────┘
Run Time (s): real 0.061 user 0.152989 sys 0.081464

Hey @mvilrokx.
I am taking the liberty to restart discussion on this issue because I am facing the same problem.
I am trying to access a Delta Lake table located on a private AWS S3 bucket. It works well when I load it using Delta Lake (delta-rs) package and then use PyArrow to read it to DuckDB (using DuckDB's Python client). But when I use this Delta - DuckDB integration, I face this error:
Error interacting with object store: Generic S3 error: Received redirect without LOCATION, this normally indicates an incorrectly configured region.
I tried setting all the types of secret possible (all combinations of REGION, CREDENTIAL_PROVIDER, and different CHAIN) but nothing works. While for the delta-rs package, I just call using the path and storage_options. Like this
storage_options = {
    "region": AWS_REGION,
}
dt = DeltaTable(path, storage_options=storage_options)

@mvilrokx I was wondering if you found a way to solve your region error.
@samansmink do you have any suggestions on what I should do to make it work?
Thank you very much in advance for your time and help. I appreciate it a lot!
Best,
Quentin

sorry for missing this one @qchappat, would you mind trying again with latest release of delta and opening a new issue if this issue persists?
I will close this issue for now since a lot was changed in the meantime

Thank you @samansmink, it seems the issue is solved now! Thanks a lot for updating me!


Code Quality: Merged all interop into Win32PInvoke

Summary
PR Checklist

[x] Were these changes approved in an issue or discussion with the project maintainers? In order to prevent extra work, feature requests and changes to the codebase must be approved before the pull request will be reviewed. This prevents extra work for the contributors and maintainers.
Closes #issue
[ ] Did you build the app and test your changes?
[ ] Did you check for accessibility? You can use Accessibility Insights for this.
[ ] Did you remove any strings from the en-us resource file?

[ ] Did you search the solution to see if the string is still being used?

[ ] Did you implement any design changes to an existing feature?

[ ] Was this change approved?

[ ] Are there any other steps that were used to validate these changes?
None

Screenshots
None

This is the last time touching around interop classes.
Deleted all interop classes and merged into single class, so we not can gradually replace them with CsWin32 without conflict (there were some classes that is the same definition) in the single place.
It's a bit large but ready for review.


Nextcloud Talk integration

See discussion here on Nextcloud Talk for integrating with Node-Red.  Cheers!

this would be very nice. maybe you found another project which can do this?


Add ingredient command logging

Part of #110

Uses .getSimpleName() instead of .getName() so that package name will not be shown
Seems repetitive (every class has its own private logger instance & the log message is explicitly written each time) but this is the current convention for logging in the app

Codecov Report

Merging #111 (630dd15) into master (89623f6) will increase coverage by 0.08%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master     #111      +/-   ##
============================================
+ Coverage     79.03%   79.11%   +0.08%
- Complexity      908      910       +2
============================================
  Files           118      118
  Lines          2552     2562      +10
  Branches        299      299
============================================
+ Hits           2017     2027      +10
  Misses          461      461
  Partials         74       74

Impacted Files
Coverage Δ

...ogic/commands/ingredient/AddIngredientCommand.java
100.00% <100.00%> (ø)

...c/commands/ingredient/DeleteIngredientCommand.java
88.23% <100.00%> (+1.56%)
:arrow_up:

...gic/commands/ingredient/EditIngredientCommand.java
93.10% <100.00%> (+0.51%)
:arrow_up:

...gic/commands/ingredient/FindIngredientCommand.java
100.00% <100.00%> (ø)

...gic/commands/ingredient/ListIngredientCommand.java
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89623f6...630dd15. Read the comment docs.


Problem to install on mac m1

I struggle with install it on mac m1, getting error related to one of dependencies, looks like architecture does not match
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [gobject_introspection.bundle] Error 1

make failed, exit code 2

seems that updating to 0.8.2 solved that issue, I can see that version does not use all those problematic gems

Hey @artur79, I'm not sure how to reproduce, but glad it's working.


cleanup bits, and --version support

I'm rather partial about the git version number and instead switching to a plain "0.7.0" which can be parsed easily. Most of the ppl that need the git version would be able to just run git log anyway, so...

done, dropped the gitversion and changed the --version output to just the version number


Update to WPCS 1.0.0

Merge in WPCS 1.0.0
Update the WordPress-Theme ruleset to account for the changes in WPCS 1.0.0

The only really relevant commit is: https://github.com/WPTRT/WordPress-Coding-Standards/commit/ed820f03f849a903324709ca3099f8477e675094 (the ruleset changes).
Once this PR has been merged, we have a good base to de-fork from WPCS.

You've been so busy!


4.0: Additional layers properties not reflected in editor or in running game

Godot version
4.0.dev 5f69218edc
System information
Kubuntu 21.04 - Lenovo Ideapad S145
Issue description
When using a tilemap in godot 4 if I add a new layer & paint tiles into the layer:

The tiles aren't rendered in the running game
The layer data is saved in the tscn file if i look at it in a text editor, but if I close and re-load the scene in the godot editor, neither the layer properties nore the previously painted tiles are displayed

Steps to reproduce
Using the minimal reproduction project:

Open Node2D.tscn
Select the TileMap node
Drop down Layers in the property inspector
Click add element
In the tiles window change to layer #1
Paint tiles from the second atlas anywhere they'd be visible
Run the project

The tiles are not rendered

Observe in the tscn file that layer_1 is defined
Close and re-open Node2D.tscn
Drop down layers in the TileMap again, observe that the layer properties are gone and it will also not be an option to switch to from the tiles sub-window
Saving at this point, gets rid of the properties from the tscn from the previous time

Minimal reproduction project
TileMapLayers.zip

cc @groud

Closing as very likely fixed by https://github.com/godotengine/godot/pull/53100.
Please let us know if the issue is still here.


[match] add option to  skip provisioning profiles

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context

We are managing the provisioning profiles separate from our certificates. This options allows for a smoother, instead of having to catch errors and ignore them if they are about provisioning profiles.

This is a personal issue. In the future we plan on moving the profiles in match as well, but thought this would be useful to others.
Description

Adds the option skip_provisioning_profile: Bool (default: false) to match

I have run this in test scripts where I am migrating all our Certificates and Private Keys to Match, and updating our build to use Match to manage Certs and Sigh to manage Profiles.
I also added a new spec.
Testing Steps

Setup a match branch where there are only certs and no profiles. Run it in read only mode and you will get an error saying profiles have to be generated. Run it again in read only and pass in skip_provisioning_profile: true and it will run successfully, no error.

@googlebot I consent.


Fully implement List Objects V2 with pagination response elements

Resolves #110. This implements "starts-after" for GET Bucket v2 as well all pagination-related response data for both the v1 and v2 APIs. This also fixes the behavior for "marker"/"starts-after" (they erroneously require a key starting with the marker to exist in order to work properly).
I also ended up doing some significant refactoring to the tests around listObjects since they weren't very strict and also ran very slowly (it actually isn't necessary to generate 1000+ objects to make adequate test cases most of the time).

@kherock Heroic effort man keep it up 💪


FS cannot be imported on Android

Describe the bug
FS module is unresolved on Android in Multiplatform builds
To Reproduce

Create a new Multiplatform project in Android Studio
Install the FS module
Import a shared library into the Android module that references FS
Build will fail with "Unresolved reference" pointing to FS module components

Expected behavior
Project should build correctly on Android
Stacktrace
If applicable, add stacktrace to help explain your problem.
Version
1.0.8
Platforms affected:

iOS [ ]
Android [x]
JVM [ ]

Additional context
Add any other context about the problem here.

Thanks for reporting this. Can you share a minimal example where I can reproduce this? Also, can this be related to gradle api vs implementation?

Closing due to inactivity. Please reopen if you are still seeing the issue.

@suparngp sorry about that I didn't get a chance to check this again, I'll try and reproduce it and reopen the ticket if I can.


node --version gives "node: command not found"

Describe the bug
After executing the instructions to install node I restart the kernel and the node --version cell fails with
/bin/sh: node: command not found
Instructions for restarting the kernel are also incorrect.
Also kernel is consistently misspelled kernal.
To Reproduce
Steps to reproduce the behavior:

Follow instructions to install nodejs
Execute cell
See error

Expected behavior
node --version should work.

I was running this in us-east-2. I restarted the workshop in us-east-1 and now it works.


Add query taint test cases

Purpose

Refactor and move testcases from ballerina-lang repo.

@mohanvive Could you send this PR from an upstream branch? Else the tokens will not be available and the build can't be verified as it fails to load dependencies

Access token not available issue is sorted with https://github.com/ballerina-platform/module-ballerina-java.jdbc/pull/27/commits/5570b9cdc2260c8c7e2fca200d938f60bc70c952

This includes fix for https://github.com/ballerina-platform/ballerina-standard-library/issues/569


How to use it with Ink

How can I use this with  Ink ?
and when running on git bash, spinner is showing only line, it not showing rotating dots link in demo, and symbols are very small, is there a way to customize it ?

By default it fallback to more uglier symbols for Windows. Thats the nature of utilized packages. Until Windows Terminal or some other customizable terminals like ConEmu or Hyper.js, it was not reliable to show symbols in Windows Terminals.
For using it with this with Ink since which also utilizes a terminal clean up renderer you have to pass it through this renderer in something like prompt function of https://github.com/cenk1cenk2/listr2/blob/master/src/lib/task-wrapper.ts . If stdout can be set in Ink. I can expose a function to utilize it.
Spinner: https://www.npmjs.com/package/elegant-spinner
Symbols:
Symbols:
https://www.npmjs.com/package/figures

I have given this a thorough try. Since I wanted to pass Ink as an output of the task.
The renderer utilizes an interval to refresh the tasks, and this INK or any other library that uses stdout utilizes ANSI escape sequences to clear out the screen, these escape sequences might be outputted more than one time depending on their rate of refresh compared to the render interval. This causes the to clear the output unexpectedly.
There are two ways to obtain this functionality:

The first one was to catch these ANSI sequences then process them in a sandbox variable and clear out the ANSI sequences, as it was done for the prompt. But prompt was a specific use-case and I did not have to implement a lot to render it properly. So this is a lot of work. Unless I find a module that can sandbox these ANSI sequences into a given variable and clear them out after processing it as told.
The second alternative is to change the render method to an event-based system instead of interval. This will require some critical structural changes to function properly albeit it is a more effective way to render stuff reactively.

So although I will give this another try since I already wanted to expose STDOUT for other functionality as output, it may take longer than expected.
I will always use the icons in the demo from now on, even though some Windows terminals may not support it.

:tada: This issue has been resolved in version 2.1.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

Please give this a try with the latest beta channel which resides here.
The section you are interested about passing the stdout through listr is here. It is by no means perfect, but it was the least I can do without creating a sandboxed terminal in the output section. It is what you call quick and dirty, albeit it may require some tweaking because it doesn't set the rows and columns properly.
Please open up a new issue for further. Looking forward for your feedback.
npm i listr2@beta
yarn add listr2@beta

:tada: This issue has been resolved in version 2.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Type Error: res.status is not a function supertest

`TypeError: res.status is not a function
  49 |     })
  50 |     .catch(error => {
> 51 |       return res.status(400).json({
     |                  ^
  52 |         success: false,
  53 |         error,
  54 |         results: null

  at status (Routes/Controllers/auth/index.js:51:18)`

router.post('/signup', (req, res) => {
let validate = ajv.compile(signupSchema);
let valid = validate(req.body);
 if (!valid) {
return res.status(400).json({
success: false,
error: sumErrors(validate.errors),
results: null
});
}
auth
 .signup(req.body)
.then(results => {
console.log(res.json);
return res.status(200).json({
success: true,
error: null,
results
});
})
.catch(error => {
return res.status(400).json({
success: false,
error,
results: null
});
});
});
Link to the project online-judge
When testing, with supertest this problem seems to exist.
But the same code works perfectly fine when I start the server manually using npm start and when requests are sent using postman.

Maybe you have to mock the response function.
One example:
beforeAll(async () => {
    const res = jest.fn(() => {
    	status: jest.fn(() => {
    		send: jest.fn()
    	})
    })
})

Maybe you have to mock the response function.
One example:
beforeAll(async () => {
    const res = jest.fn(() => {
    	status: jest.fn(() => {
    		send: jest.fn()
    	})
    })
})

But is'nt this supposed to work even without mocking..
I have the link for the project if u want
https://github.com/NBNARADHYA/online-judge

@NBNARADHYA are you still experiencing this issue? How exactly are you using supertest?

It happens when you try to test the express.Router() instance.
Using express() instead of express.Router() solved my problem.

Can you explain why express() works differently from express.Router() when using supertest?


Libvirt security updates

See #6416
Edit:
There's a <IP_ADDRESS> maintenance release that is meant to fix the CVE's listed in the previous PR. Should we push the update to 1.2.12 or use the maintenance release for stable?

Looking at the changelog[1] between 1.2.9 and 1.2.12, although it is very long and I couldn't read all, the entries under "new features" are very few. And they don't mention any backwards compatibility issues (in the changelog at least). The other sections in the changelog are "security", "documentation", "improvements", "bug fixes". So, it seems safe enough to go to 1.2.12, IMHO.
But then again, why did they make <IP_ADDRESS>, if 1.2.12 seems so safe? :-)
Btw, for backporting commits, Eelco prefers "cherry-pick -x" (which I completely understand). (I can do that myself, it's just FYI.)
1: http://libvirt.org/news.html

Ah, yeah cherry-pick -x makes sense. :)

Oh, you closed it? I thought it was a good idea to keep it open for discussions. (And now I cannot re-open because the branch is gone.)
So... cherry-pick or use the tiny bugfix-release <IP_ADDRESS>? If nobody objects, I think I'll go for the cherry-pick.

Ah, sorry, figured you'ld decided to use the 1.2.12 release. :)
I don't mind

Fails to build, see travis error. Please enable nix chroot builds: http://nixos.org/nixos/manual/ch-options.html#opt-nix.useChroot

@lethalman I do use nix chroot builds. This doesn't fail to build locally for me, so I'm confused by the travis error.

Built locally, thanks.


fix(slider): high contrast improvements for filled track

Description
Filled sections of the track get the Highlight color when in high contrast mode to differentiate them from the rest of the track, which remains the ButtonText color (even on hover/focus/keyboard focus/active state). Slack receipts in this thread.
How and where has this been tested?
Please tag yourself on the tests you've marked complete to confirm the tests have been run by someone other than the author.
Validation steps

[x] Design approval (@mdt2)
[x] Open the Slider in the docs site and turn on high contrast @jenndiaz
[x] The filled portion of the track should be the Highlight color to differentiate them from the rest of the track for: @jenndiaz

[x] Filled
[x] Filled-offset
[x] Range

Regression testing
Validate:

The documentation pages for at least two other components are still loading, including:

[ ] The pages render correctly, are accessible, and are responsive.

If components have been modified, VRTs have been run on this branch:

[ ] VRTs have been run and looked at.
[ ] Any VRT changes have been accepted (by reviewer and/or PR author), or there are no changes.

To-do list

[x] I have read the contribution guidelines.
[ ] I have updated relevant storybook stories and templates.
[x] I have tested these changes in Windows High Contrast mode.
[ ] If my change impacts other components, I have tested to make sure they don't break.
[ ] If my change impacts documentation, I have updated the documentation accordingly.
[ ] ✨ This pull request is ready to merge. ✨

Beta released<EMAIL_ADDRESS>


Problems with class loading in Rails production (like) environments

Hi,
there was a little issue with class loading (when processes/workers/threads are restarted i guess) that sometimes the workers did not load the rails environment properly.
Some errors like "circular dependency" or that methods on classes/objects are not found that should obviously be there.
It only happens in a environment where classes are cached (rails production or when otherwise activated).
The fix is to make sure the eager-loading the environment is manually triggered (which is deactivated by default for rake tasks).
Just add this line:
::Rails.application.eager_load!

Before doing anything else in the sneakers:run rake task and it works fine (did so far here).

I'm curious why this isn't built into sneakers:run rake task provided by the sneakers gem?

+1 for this being built into sneakers. It caused problems with our production. Could you reopen?

@sheerun: probably easiest to re-open as a pull-request. Just a thought :)

Where did you add this line of code? In your own Rake task executing the sneakers:run task or in Rakefile?
I get "LoadError: No such file to load -- application_controller" all the time.

I created a lib/tasks/sneakers.rake to call this and then run the requested rake task

Me too. Had to let the task depend on environment to not get the LoadError though. Don't know if this is correct but I'll try it out anyway.
task run2: :environment do
  ::Rails.application.eager_load!
  Rake::Task['sneakers:run'].invoke
end

Just in case anyone lands here and wonders if it was added to the default rake task, it was on 2015.04.01: https://github.com/jondot/sneakers/commit/4f07a836fd1142c46702ea5267d4291d180c6da3


Bugged Text

It's only happened on YouTube, but it's being buggy. A video started with "Welcome to [DEADNAME] industries! Fortnite #41 | Solo Battles" then the actual title was after it.

are you gonna reply to this?

Hi @Diyamund!
Are you using the latest version of DeadName-Remover?
Regards,
Gusted

When was it last updated?

When was it last updated?

2 Weeks ago


Release/v0.1.22

New functionality Retry
Rename functions RequestRepeat* to RequeRetry*
Fix broken tests

@samybenatt hello. I hope you are doing well. Could you please review this PR?


docs(highlights): add highlights for 0.26.0

This looks great! Thanks for writing it up!
I think we should mention theFill constraint in a separate section in addition to flex mode changes.
I can update the flex section with some more details and link to another page that has even more information, if you want to merge as is.

Feel free to push to this PR if you want to make changes or we can just merge it! (But we probably can get the opinions of Josh and Valentin before that.)

I was trying to add this gif but it doesn't seem to be rendering locally for me. I've just pushed it using git lfs:


Enas Tutorial is not up-to-date

I can not run the https://github.com/microsoft/nni/blob/v2.10/examples/nas/oneshot/enas/search.py
it give me error on this one from nni.nas.pytorch import mutables to indicate no mutables in nni.nas.pytorch, when I change it, it give me errors because the MutableScope(See https://github.com/microsoft/nni/blob/c31d2574cb418acfb80c17bb2bd03531556325bd/examples/nas/oneshot/enas/micro.py#L62) is deleted since v2.9
Environment:

NNI version: v2.10
Training service (local):
Client OS: Ubuntu20.04
Python version: 3.10
PyTorch version: 1.10+cu113
Is conda/virtualenv/venv used?: No
Is running in Docker?: Yes

Configuration:

Search space: ENAS

Please check the latest documents: https://nni.readthedocs.io/en/stable/tutorials/darts.html
We might need to delete the old examples at some point...

Please check the latest documents: https://nni.readthedocs.io/en/stable/tutorials/darts.html
We might need to delete the old examples at some point...

so just repace "strategy = DartsStrategy" with  "strategy = DartsStrategy"  will be ok?

is there an example showing useage of enas  that can run without modification now?

so just repace "strategy = DartsStrategy" with  "strategy = ENAStrategy"  will be ok?

I suppose so.

is there any example showing useage of enas  that can run without modification now?

Maybe you can find one in unit-tests. But they are not meant for beginners.

I think if I follow the tutorial, then the search space need to be changed as well. But in ENAS search space, it is hard for me to set up ENAS for there is no instruction to build micro or macro search space

Up to now, I can only follow dart tutorial to run one shot NAS. I need to have torchmetrics == 0.10 and numpy == 1.23.5 for nni == 2.10. Enas seems to need have some controller settings as well or use EnasTrainer

@ultmaster Thanks for the tutorial. In the example tutorial, in the end it concluded that with cutout for data transform, the final performance of trained model is better. And I wonder if the cutout should apply to NAS search and model training or just model training?

@ultmaster  I think if I follow the tutorial, then the search space need to be changed as well. But in ENAS search space, it is hard for me to set up for there is no instructions to build micro or macro search space

So you are trying to reproduce ENAS search space? Reproducing ENAS search space and ENAS search strategy are two independent things.

@ultmaster Thanks for the tutorial. In the example tutorial, in the end it concluded that with cutout for data transform, the final performance of trained model is better. And I wonder if the cutout should apply to NAS search and model training or just model training?

You can use cutout at training, but the original authors didn't. I suspect it's due to efficiency reasons.

any updates or close? @dzk9528

@Lijiaoa Well, since DARTS example work well(96.88% in the end), I think it is good enough for me at this point. I will not stick to ENAS tutorial, so it depends on @ultmaster and to update the doc, which definitely take a lot of effort and time. I think I will close this issue tomorrow.

import nni
import numpy as np
from nni.retiarii.evaluator.pytorch import Classification
from torch.utils.data import SubsetRandomSampler
from torchvision import transforms
from torchvision.datasets import CIFAR10
from nni.retiarii.evaluator.pytorch import DataLoader
from nni.retiarii.experiment.pytorch import RetiariiExperiment, RetiariiExeConfig
from nni.retiarii.strategy.rl  import PolicyBasedRL
from nni.retiarii import fixed_arch
from nni.retiarii.strategy import ENAS as ENAStrategy
from nni.retiarii.hub.pytorch import ENAS as ENASpace
from nni.retiarii.strategy import DARTS as DartsStrategy
from nni.retiarii.hub.pytorch import DARTS as DartsSpace
CIFAR_MEAN = [0.49139968, 0.48215827, 0.44653124]
CIFAR_STD = [0.24703233, 0.24348505, 0.26158768]
fast_dev_run = True
fast_dev_run = False
strategy = DartsStrategy()
strategy = ENAStrategy(reward_metric_name='val_acc')
transform = transforms.Compose([
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(CIFAR_MEAN, CIFAR_STD),
])
train_data = nni.trace(CIFAR10)(root='./data', train=True, download=True, transform=transform)
num_samples = len(train_data)
indices = np.random.permutation(num_samples)
split = num_samples // 2
search_train_loader = DataLoader(
train_data, batch_size=64, num_workers=6,
sampler=SubsetRandomSampler(indices[:split]),
)
search_valid_loader = DataLoader(
train_data, batch_size=64, num_workers=6,
sampler=SubsetRandomSampler(indices[split:]),
)
evaluator = Classification(
learning_rate=1e-3,
weight_decay=1e-4,
train_dataloaders=search_train_loader,
val_dataloaders=search_valid_loader,
# max_epochs=10,
max_epochs=1,
gpus=1,
fast_dev_run=fast_dev_run,
)
model_space = DartsSpace(
width=16,           # the initial filters (channel number) for the model
num_cells=8,        # the number of stacked cells in total
dataset='cifar'     # to give a hint about input resolution, here is 32x32
)
model_space = ENASpace(
width=16,           # the initial filters (channel number) for the model
num_cells=8,        # the number of stacked cells in total
dataset='cifar'     # to give a hint about input resolution, here is 32x32
)
config = RetiariiExeConfig(execution_engine='oneshot')
experiment = RetiariiExperiment(model_space, evaluator=evaluator, strategy=strategy)
experiment.run(config)
trategy.run()
exported_arch = experiment.export_top_models('dict')[0]
print(exported_arch)
strategy.export_top_models()
with fixed_arch(exported_arch):
# final_model = DartsSpace(width=16, num_cells=8, dataset='cifar')
final_model = ENASpace(width=16, num_cells=8, dataset='cifar')
train_loader = DataLoader(train_data, batch_size=96, num_workers=6)  # Use the original training data
max_epochs = 1
evaluator = Classification(
learning_rate=1e-3,
weight_decay=1e-4,
train_dataloaders=train_loader,
val_dataloaders=search_valid_loader,
max_epochs=max_epochs,
gpus=1,
export_onnx=False,          # Disable ONNX export for this experiment
fast_dev_run=fast_dev_run   # Should be false for fully training
)
evaluator.fit(final_model)
this is my enas code,I don't know if there is any error while it can run. I paste it here for these in need.

```python
import nni
import numpy as np
from nni.retiarii.evaluator.pytorch import Classification
from torch.utils.data import SubsetRandomSampler
from torchvision import transforms
from torchvision.datasets import CIFAR10
from nni.retiarii.evaluator.pytorch import DataLoader

from nni.retiarii.experiment.pytorch import RetiariiExperiment, RetiariiExeConfig
from nni.retiarii.strategy.rl  import PolicyBasedRL
from nni.retiarii import fixed_arch

from nni.retiarii.strategy import ENAS as ENAStrategy
from nni.retiarii.hub.pytorch import ENAS as ENASpace

from nni.retiarii.strategy import DARTS as DartsStrategy
from nni.retiarii.hub.pytorch import DARTS as DartsSpace

CIFAR_MEAN = [0.49139968, 0.48215827, 0.44653124]
CIFAR_STD = [0.24703233, 0.24348505, 0.26158768]

# fast_dev_run = True
fast_dev_run = False

# strategy = DartsStrategy()
strategy = ENAStrategy(reward_metric_name='val_acc')

transform = transforms.Compose([
    transforms.RandomCrop(32, padding=4),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize(CIFAR_MEAN, CIFAR_STD),
])

train_data = nni.trace(CIFAR10)(root='./data', train=True, download=True, transform=transform)

num_samples = len(train_data)
indices = np.random.permutation(num_samples)
split = num_samples // 2

search_train_loader = DataLoader(
    train_data, batch_size=64, num_workers=6,
    sampler=SubsetRandomSampler(indices[:split]),
)

search_valid_loader = DataLoader(
    train_data, batch_size=64, num_workers=6,
    sampler=SubsetRandomSampler(indices[split:]),
)

evaluator = Classification(
    learning_rate=1e-3,
    weight_decay=1e-4,
    train_dataloaders=search_train_loader,
    val_dataloaders=search_valid_loader,
    # max_epochs=10,
    max_epochs=1,
    gpus=1,
    fast_dev_run=fast_dev_run,
)

# model_space = DartsSpace(
#     width=16,           # the initial filters (channel number) for the model
#     num_cells=8,        # the number of stacked cells in total
#     dataset='cifar'     # to give a hint about input resolution, here is 32x32
# )

model_space = ENASpace(
    width=16,           # the initial filters (channel number) for the model
    num_cells=8,        # the number of stacked cells in total
    dataset='cifar'     # to give a hint about input resolution, here is 32x32
)

config = RetiariiExeConfig(execution_engine='oneshot')
experiment = RetiariiExperiment(model_space, evaluator=evaluator, strategy=strategy)
experiment.run(config)
# trategy.run()
exported_arch = experiment.export_top_models('dict')[0]
print(exported_arch)
# strategy.export_top_models()

with fixed_arch(exported_arch):
    # final_model = DartsSpace(width=16, num_cells=8, dataset='cifar')
    final_model = ENASpace(width=16, num_cells=8, dataset='cifar')
train_loader = DataLoader(train_data, batch_size=96, num_workers=6)  # Use the original training data

max_epochs = 1

evaluator = Classification(
    learning_rate=1e-3,
    weight_decay=1e-4,
    train_dataloaders=train_loader,
    val_dataloaders=search_valid_loader,
    max_epochs=max_epochs,
    gpus=1,
    export_onnx=False,          # Disable ONNX export for this experiment
    fast_dev_run=fast_dev_run   # Should be false for fully training
)

evaluator.fit(final_model)

this is my enas code,I don't know if there is any error while it can run. I paste it here for these in need.

good job!


File upload issue

What is your Test Scenario?
an input type file but is not visible
What is the Current behavior?
the attribute file and the value are loaded with the data but the file is not being uploaded
What is the Expected behavior?
Upload the file
What is your web application and your TestCafe test code?
test cafe code:
.setFilesToUpload(myAccountPage.uploadFileInput, '../../mock-files/originalpicture.png')
the locator : this.uploadFileInput = Selector('#file-upload');
the html:

Your Environment details:

testcafe version:  0.22

node.js version:  v8.11.4

browser name and version: Chrome Version 74.0.3729.131 (Official Build) (64-bit)

platform and version:          Windows 10

Please note that the setFilesToUpload and clearUpload actions only allow you to manage the list of files for upload.
These files are uploaded to the server after you initiate an upload, for example, when you click the Upload or Submit button on a webpage. For example
import { Selector, ClientFunction } from 'testcafe';

fixture`fixture`
    .page`http://localhost/site/`;

test('test', async t => {
    const fileInput = Selector('input[type=file]');

    const getUploadedFileName = ClientFunction(() => {
        const input = fileInput();

        if (!input.files.length)
            return;

        return input.files[0].name;
    }, { dependencies: { fileInput } });

    await t
        .expect(getUploadedFileName()).notOk()
        .setFilesToUpload(fileInput, './_uploads/image.png')

	//after using setFilesToUpload, you can check an input's file attribute
        .expect(getUploadedFileName()).eql('image.png')

	//but you can check the real file upload only after submitting a form
        .click('#submit');
});

@helen-dikareva I'm working with Matias on this one. After some more research, we found that the input is being filled with the file list, the problem is that the front end is not responding in the same way with testcafé, as with manual testing.
Could it be that changing the value of the file input is not triggering some event that the front-end needs?
Are you aware of some limitation on testcafé regarding the file upload that we should be aware of?
I know that using the OS window is not possible because it is outside of the browser. I believe our issue has to be related to some event not being fired.
On a somewhat related question. Is there a way inside of testcafé to force / trigger some event?
Thanks for your help. It's much appreciated as usual

@berrutti

After some more research, we found that the input is being filled with the file list, the problem is that the front end is not responding in the same way with testcafé, as with manual testing.

The behavior should be the same.
I see that you are using a very old TestCafe version - 0.22.
Try to update to the latest version - 1.1.4. If the problem still persists, please provide us with a simple example. In this case, we can investigate the problem faster and find a precise solution.


Adding serverless ai hello world to v2 folder

This will need some sidebar surgery also @karthik2804
This new page was added to the standard docs after the cp spin/ v2 was performed.
Will there be a separate spin_sidebar.hbs file for v2?

@karthik2804 ’s PR already added this page so this can be close.


Validating hierarchical data structures of unknown depth

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
 pydantic version: 1.5.1
            pydantic compiled: True
                 install path: /Users/cflynn15/PycharmProjects/cara/venv/lib/python3.7/site-packages/pydantic
               python version: 3.7.4 (default, Oct 12 2019, 18:55:28)  [Clang 11.0.0 (clang-1<IP_ADDRESS>)]
                     platform: Darwin-18.7.0-x86_64-i386-64bit
     optional deps. installed: []

Question
Hello! I'm new to pydantic and am trying to figure out the best way to use a pydantic model as a response_model within fast-api. The tricky part is that the object essentially a hierarchical tree where each node has the same attributes.

I've read the docs on post annotations and self-referencing models but am not sure how to/If I should apply them in this case.
The data structure I want to validate looks like this:
{
	"orgid": 1,
	"name": "SomeRootOrg",
	"parentid": -1,
	"level": 0,
	"id": 0,
	"path": [
		0
	],
	"children": [{
			"orgid": 2,
			"name": "SomeChildOrg",
			"parentid": 1,
			"level": 1,
			"id": 1000,
			"path": [
				0,
				1000
			],
			"children": [{
					"orgid": 13,
					"name": "AnotherChild",
					"parentid": 2,
					"level": 2,
					"id": 2000,
					"path": [
						0,
						1000,
						2000
					],
					"children": [{...AndSoOn...}]
}

Where the maximum depth would be 5 at this point in time. I wrote a pydantic model which can successfully validate this object however there's always the potential for org changes in the future where a new level could potentially be added. Any advice on how or If this could/should be validated through pydantic is welcome. Thanks!

from typing import List, Optional
from pydantic import BaseModel

class BaseOrg(BaseModel):
    orgid: int
    name: str
    parentid: int
    level: int
    id: int
    path: List[int]

class Level1Org(BaseOrg):
    children: Optional[List[BaseOrg]]

class Level2Org(Level1Org):
    children: Optional[List[Level1Org]]

class Level3Org(Level2Org):
    children: Optional[List[Level2Org]]

class Level4Org(Level3Org):
    children: Optional[List[Level3Org]]

class Level5Org(Level4Org):
    children: Optional[List[Level4Org]]

...

I would say self-referencing models and maybe validators would be the best way to go, or what you have above and worry about something more complex when you need it.

this is very simple
class Node(BaseModel):     title: str     children: List['Node']
yes, just use string 'Node', python can smart find it


feat(vite)!: svelte-scoped automatic compilation

Closes #1676
I've written test cases for every situation described in #1676 and a few less common ones as well. As can be seen by running the tests and the updated svelte-scoped example project, the SvelteScopedPlugin is now fully ready for use in exactly the same manner as is available using global styles, except that each component carries the weight of its own CSS.
Merging this will prepare the way for #1719 and #1718.

I put this back into draft mode as I've been using this in production via my own temporarily published packaged (#1725), and I made a couple changes that I want to document before merge.

@antfu Can you please take another look at this to see if it's good to merge or give advice on what to change?
I'd love to help with some of the other SvelteKit related issues like #1774 as well as dig into further explorations of how to smoothly integrate the Inspector and typography plugin in a context where a global stylesheet is not the default place to put all styles. But I think this PR is already big enough.


Revert "Initial implementation of Command Plugins"

Reverts apple/swift-package-manager#3855

@swift-ci please test

@compnerd Is there a log where I can see how this failed?  It should be possible to fix this forward.

@abertelrud @compnerd is this addressed by #3888 and can be close?

@tomerd - I want to keep this open until we know that it does fix the build, otherwise, I think that we should roll back to get the builds green again - they have been broken for ~12 hours now.

@abertelrud @compnerd is this addressed by #3888 and can be close?

In my view we can close this, since I'm confident that we can fix this forward.  We are still testing #3888 but I don't see a circumstance in which we'd need to revert this commit.


Support for including utterances/ samples in slots

Now the schema's slot is an object that takes a map. Doc says "The slots object is a simple name: type mapping. The type must be one of Amazon's built-in slot types, such as AMAZON.DATE or AMAZON.NUMBER."
If we require to specify a list of samples, for a slot it is not possible. This is especially useful when modeling a dialog flow. see https://raw.githubusercontent.com/alexa/alexa-cookbook/master/feature-demos/skill-demo-plan-my-trip/models/en-IN.json
In code it is explicitly passed as empty array. Changing the value of the map from string to object should do.

if (intent.slots && Object.keys(intent.slots).length > 0) {
        intentSchema["slots"] = [];
        for (key in intent.slots) {
          //  It's unclear whether `samples` is actually used for slots,
          // but the interaction model will not build without an (empty) array
          intentSchema.slots.push({
            "name": key,
            "type": intent.slots[key],
            "samples": []
          });
        }
      }
      schema.intents.push(intentSchema);
    }

Care to pull request a fix for this?

Comment added on the PR. Merge and release soon?


error

You made a mistake, in comfig.yml you can not change the time of the explosion

Fixed Please use Latest version
here: https://poggit.pmmp.io/p/TNTExplodeTime/1.0.1


Get meta-data for dataset, not just file

In order to speed up retrieval of meta-data (e.g., confidence scores) and retrieve the number of API calls, an API end point to retrieve all meta-data associated with a dataset should be available. The existing end point for retrieving the meta-data for a file of a dataset could be repurposed for that: by leaving out the file name, all of the meta-data is returned (with the additional layer of file names, of course).
Current end point:
GET: /v1/datasets/{PK}/metadata/{NAME}
Should become:
GET: /v1/datasets/{PK}/metadata[/{NAME}]

Implemented in:

f0c1d44544cff19148d9de61968b718b193b5a0a
ufdl-python-client

The JSON returned by the bulk end-point is:
{
  "filename1": "meta-data for file 1",
  "filename2": "meta-data for file 2",
  ...
}


Some areas have no blur effect

There is no blur effect in the control bar and lyrics area
It's completely transparent.

Did i set something wrong?

can you confirm this is only present on iTheme? can you try base Cider and see if the issue still persists there?

It looks normal after the app update.Thank you for your reply.
发自我的iPhone
------------------ Original ------------------
From: KillerParticles @.>
Date: Wed,Jul 20,2022 8:31 PM
To: ciderapp/iTheme @.>
Cc: SilentVastSky @.>, Author @.>
Subject: Re: [ciderapp/iTheme] Some areas have no blur effect (Issue #54)
can you confirm this is only present on iTheme? can you try base Cider and see if the issue still persists there?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

closing aa issue is resolved


SharedStore provider

Motivation
For additional context see: https://github.com/kube-rs/kube/pull/1128
In large controllers you can end up with multiple instances of the same store, this has the potential to case a split brain problem where different bits of the application have a different view of the cluster.
This can be solved by creating stores at the top level of the application and passing them down; but the definition of the stores is then required to be at a distance from the usage sites, smearing responsibilities around the codebase.
Solution
The SharedStore struct will always return a clone of the same Store for a given scope and ListParams. The aim is to minimise the knowledge that must exist at a distance (only the types must be known, which can be cleanly asked for in controller interfaces), while still allowing separate parts of the application to share stores.

@clux I've pulled out just the SharedStore bit of my bigger PR (selfishly because this is the most useful bit for us). Obviously we're missing docs and some tests here but before I spend more time on it I wanted to discuss if this is a direction worth pursuing.

Codecov Report

Merging #1147 (7c1cee1) into main (53ad9ee) will decrease coverage by 1.00%.
The diff coverage is 0.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1147      +/-   ##
==========================================
- Coverage   72.79%   71.80%   -1.00%
==========================================
  Files          66       68       +2
  Lines        5066     5136      +70
==========================================
  Hits         3688     3688
- Misses       1378     1448      +70

Impacted Files
Coverage Δ

kube-core/src/params.rs
78.52% <ø> (ø)

kube-runtime/src/reflector/mod.rs
100.00% <ø> (ø)

kube-runtime/src/reflector/shared_store.rs
0.00% <0.00%> (ø)

kube-runtime/src/utils/mod.rs
64.15% <ø> (ø)

kube-runtime/src/utils/stream_subscribe.rs
0.00% <0.00%> (ø)

kube-runtime/src/utils/watch_ext.rs
0.00% <0.00%> (ø)

I think the basic idea of this struct is good, but it's harder to see where to go optimally here without seeing how you hook it up to a controller (because it doesn't help much to generate stores up front when Controller ultimately creates their own stores).
The potential setup you have here can work, but I can also see it having potential as a more global stream manager that maintains watcher streams for all types regardless of kind. The latter might be an easier thing to integrate into Controller as an endgame for #1080. At any rate, would be interested in hearing your thoughts on integration here.

I had a go at creating a generic

I think the basic idea of this struct is good, but it's harder to see where to go optimally here without seeing how you hook it up to a controller (because it doesn't help much to generate stores up front when Controller ultimately creates their own stores).

I haven't had chance to play around with this yet, but the idea in my head is something along the lines of:
let lp = ListParams::Default();
// I'm still in two minds if the SharedStore functions should just always return a store and a stream and you just drop
// one if you don't care about it
let store = shared_store.all(lp);
let stream = shared_store.subscribe_all(lp);

let other_stream = other_shared_store().namespaced_subscribe(lp);

Controller::new_from_store(store stream)
     .watches_stream(other_stream, |thing| {
            Some(ObjectRef::new(&thing.name_unchecked()))
     });

The potential setup you have here can work, but I can also see it having potential as a more global stream manager that maintains watcher streams for all types regardless of kind. The latter might be an easier thing to integrate into Controller as an endgame for #1080. At any rate, would be interested in hearing your thoughts on integration here.

I did have a go at creating an "all types" version of this, but I couldn't think of a way of creating it without losing compile time type safety. The only way I can think of would be to change the reflectors property to be indexed by something like type ScopedListParams = (ApiResource, Option<String>, ListParams); then strongly typing them on the way out to the user. That feels horrible to me, but I guess the streams will always come into the Rust untyped and must be cast to a known type at some point.

Just wanted to drive by and say this isn't forgotten about, I just haven't had time to work on it recently. Hopefully I'll be able to bring some focus back to it soon.

Appreciate it! We are early-stage exploring the other side of the coin here with the controller interface. See https://github.com/kube-rs/kube/pull/1163 for some context.

Hey @danrspencer , I know this might not be on your radar, just wondering what you ended up doing or if you still have interest in this.
It is certain that we still need at least two more things to close out #1080;

waiting for reflectors to fully init before starting the controller
hooking up your streamsubscribe (that has made it in under unstable in your other PR) through some arc-wrapping

if you have solutions elsewhere it'd be good to know about, and i'll open other issues

Hey,
Yep it’s still on my radar. The work I was doing that I could consider controller improvements part of got punted down the backlog but we should be picking it up again soon. We’ve got at least a couple of concrete cases in our codebase where not having shared stores is causing race condition bugs so I should be able to find some time to get back to this as part of solving those.

@clux I'm going to close this PR. I'm moving jobs and realistically I'm probably not going to get chance to pick this up again. Thanks for your help and feedback getting it this far. Hopefully some of the ideas and discussions will help solve the problem.


Differentiate agent lifecycle vs job lifecycle hooks

Presently, the term agent hooks can be used to refer to either the buildkite-agent feature or the location that hooks are stored in the file system.
With the new pre-bootstrap and arguably agent-shutdown hooks we have introduced a new category of hook, one that is run outside the step/job lifecycle and that is executed rather than sourced.
I’d like to make that distinction clear and think introducing new terminology is the way to do that.
This feels like a significant change to the product area. ~I’m not tied to this particular terminology or this particular slicing of things, but I wanted to have something written down for us to discuss pros and cons 😄~ Edit: I have since discussed this with others and am fairly tied to this new naming. How I’ve named and sliced this up is elaborated on 👇 and the doc diff reflects this.

This change continues to use the term agent hooks to refer to the all encompassing feature of the Buildkite Agent.
There are two categories of agent hook: Agent Lifecycle Hooks and Job Lifecycle Hooks.

Agent Lifecycle Hooks are executed as part of the agent lifecycle, presently pre-bootstrap and agent-shutdown.
Job Lifecycle Hooks (the new name for the old feature) are sourced as part of the bootstrap lifecycle, all our old favourites environment, pre-command, post-artifact etc

There are three locations that agent hooks are defined:

Agent hooks (the new name for the old location) are stored on the agent host. This location can define both agent lifecycle hooks, and job lifecycle hooks.
Repository hooks, as before, can define job lifecycle hooks
Plugin hooks, as before can define job lifecycle hooks

Next steps:

[x] Update strict checks example to use BUILDKITE_ENV_FILE for vetting rather than env vars
[x] Reach consensus on naming
[x] Find other places to update naming

This is great stuff. Did you discuss "types" as an alternative to "categories", and was it decided against because of the multiple hooks in Lifecycle hooks?

I went with category because we already use type to refer to the existing collection of hooks and I wanted to preserve that while introducing a new grouping. This way we have two categories, and in those two categories we have multiple types. What do you think?

How do you feel compacting that down to something like

I think these suggested header changes are great, I’ll incorporate those now 😄

I'd also like a super short section on "in July 2021 we slightly changed how we refer to Agent hooks, although we could fold that into the changelog (a better approach or even both).

I think both is good, I’ll work that into the intro too :+1:

What do you think of the relative ordering of "Agent hook locations" in the doc? It was higher but I pushed it to the end, maybe it should come before the discussion of the different hook categories and types though?

I went with category because we already use type to refer to the existing collection of hooks and I wanted to preserve that while introducing a new grouping. This way we have two categories, and in those two categories we have multiple types. What do you think?

Re-reviewing this we used to use "type" to refer to the location of the hook rather than the hook’s function 🙈
I do still think this new naming names makes sense though, that there are two categories across three locations, in those locations we have different hook types.

I do still think this new naming names makes sense though, that there are two categories across three locations, in those locations we have different hook types.

The question I ask myself here is "even if the type / category thing is technically correct, does it matter to the read" and I think here it does make sense to make a distinction.

What do you think of the relative ordering of "Agent hook locations" in the doc? It was higher but I pushed it to the end, maybe it should come before the discussion of the different hook categories and types though?

As I see it there are now two minor awkward parts to the doc. The repetition of hook categories follow by the same info in the toc one para down, and the hook location, especially where we're mentioning them in the table.
How about we just link all the hook locations in the table to the relevant section?
And ignore the two categories coming before doc as not a real problem.

@keithduncan where are we at with this? Still deciding on the exact naming?

We’re ready to go with this naming @plaindocs!