Add compatibility to make integration into Torchnet simpler

Facebook's Torchnet (just released) has it's own implemented dataset solution. It lacks a csv-interface, handling of categories, and core statistics. It is from what I understand mostly an approach to sampling that requires functions to have two functions implemented:

dataset:size() which returns the size of the dataset.
dataset:get(idx) where idx is a number between 1 and the dataset size.

This would require changing the Dataframe:size() function in the develop branch to return only  number rows instead of both rows and columns. The get() is synonymous with get_row() if I understand this correctly:

In torchnet, a sample returned by dataset:get() is supposed to be a Lua table. Fields of the table can be arbitrary, even though many datasets will only work with torch tensors.

The latter sentence suggests that changing the internal storage (issue #16) may be wise for optimal integration.

The update is closing in a mutch more mature state. I'm struggling with how to approach the parallel dataset iterator. In the mnist example it is rather straight forward:
-- function that sets of dataset iterator:
local function getIterator(mode)
   return tnt.ParallelDatasetIterator{
      nthread = 1,
      init    = function() require 'torchnet' end,
      closure = function()

         -- load MNIST dataset:
         local mnist = require 'mnist'
         local dataset = mnist[mode .. 'dataset']()
         dataset.data = dataset.data:reshape(dataset.data:size(1),
            dataset.data:size(2) * dataset.data:size(3)):double()

         -- return batches of data:
         return tnt.BatchDataset{
            batchsize = 128,
            dataset = tnt.ListDataset{  -- replace this by your own dataset
               list = torch.range(1, dataset.data:size(1)):long(),
               load = function(idx)
                  return {
                     input  = dataset.data[idx],
                     target = torch.LongTensor{dataset.label[idx] + 1},
                  }  -- sample contains input and target
               end,
            }
         }
      end,
   }
end

The problem here is that they use the mnist package for loading the actual data into each thread and then pass a global index to the batch dataset. The sampler functions that we use from the Twitter dataset keep an internal index. We could modify so that:

the enque function calls get_batch before the threads:addjob(
the batch is then serialized and the batch frame replaces the idx in arglist
replace the get method with the to_tensor (alt. override the Batchframe inherited get with a specific get method for the Batchframe
modify how the transform/filter work and add the target_/input_transform. The latter two also need to be exported into the thread environment


reset is triggered 2 times on k-redux-factory types

Because

"addReset"  self dispatch its action
"toContext" dispatch k-redux-factory actions listed here: https://github.com/alakarteio/k-ramel/blob/master/packages/k-ramel/src/createStore/toContext.js#L14

in result we have dispatch(dispatch(reset(path))

the solution is to remove the "reset" from both arrays in toContext file


Use a loading image in "src"

Hello Verlok, and thank you for this very useful plugin!
I would like to ask you a question about 'placeholders'!
I use lazyload on my site but with a 'placeholder' in 'src', I know you do not recommend it, but I want to know if it only affects the loading time or if it has a negative effect on the SEO!?
If it's possible, I'd like to keep it that way! (with a placeholder in the 'src' attribute).
Thank you.

Duplicate of #215

Hello @moon-lover,
I’m still investigating on how lazy loading affects SEO, after they opened #215.

Thanks for the answer verlok!


Feat cookies support

Features

Added resp.Cookies() method to ensure access response cookies
Updated golang version and dependency versions

Fix

Timeout error caused by test isolation has been fixed on response tests

Testing

Added ci-cd workflows for testing
Implemented res.Cookies() method tests

Note: This pr related with #2

I have been added cookie support for responses, can you review my pr? Also I added workflows into ci-cd for testing. This pr related with #2, I reopened for now @yakuter


Ability to pass inputs to custom widget programmatically

View:
  onLoad: |-
    //@code
    myCustomWidget.inputs.data = "my data";

  Column:
    children:
      - CustomWidget:
          id: myCustomWidget

CustomWidget:
  inputs:
    - data
  Text:
     text: ${data}

the actual issue is doing in the onLoad method.

@Ammar-Ishfaq - can you provide a full use case?

still doesn't work, test case here - https://studio.ensembleui.com/app/a2qNNXgcmhBxfrTMBVGB/screen/iEY1PEkTWUzyGUs3U5mZ
I will work on it.

still doesn't work. Assigning to @vusters to be re-assigned


Don't send more events once deletion timestamp has been set

Description
Pod deletion workflow is as follows
UPDATE
6c6
<     "resourceVersion": "28821",
---
>     "resourceVersion": "28866",
7a8,9
>     "deletionTimestamp": "2022-07-12T23:56:13Z",
>     "deletionGracePeriodSeconds": 30,

UPDATE
6c6
<     "resourceVersion": "28866",
---
>     "resourceVersion": "28868",
15,16c15,16
<       "cni.projectcalico.org/podIP": "<IP_ADDRESS>/32",
<       "cni.projectcalico.org/podIPs": "<IP_ADDRESS>/32"
---
>       "cni.projectcalico.org/podIP": "",
>       "cni.projectcalico.org/podIPs": ""

UPDATE
6c6
<     "resourceVersion": "28868",
---
>     "resourceVersion": "28873",
74c74
<         "time": "2022-07-12T23:55:36Z",
---
>         "time": "2022-07-12T23:55:44Z",
82a83,84
>                 "f:message": {},
>                 "f:reason": {},
96a99,100
>                 "f:message": {},
>                 "f:reason": {},
104,111d107
<             "f:podIP": {},
<             "f:podIPs": {
<               ".": {},
<               "k:{\"ip\":\"<IP_ADDRESS>\"}": {
<                 ".": {},
<                 "f:ip": {}
<               }
<             },
213c209
<         "status": "True",
---
>         "status": "False",
215c211,213
<         "lastTransitionTime": "2022-07-12T23:55:36Z"
---
>         "lastTransitionTime": "2022-07-12T23:55:44Z",
>         "reason": "ContainersNotReady",
>         "message": "containers with unready status: [nginx]"
219c217
<         "status": "True",
---
>         "status": "False",
221c219,221
<         "lastTransitionTime": "2022-07-12T23:55:36Z"
---
>         "lastTransitionTime": "2022-07-12T23:55:44Z",
>         "reason": "ContainersNotReady",
>         "message": "containers with unready status: [nginx]"
231,236d230
<     "podIP": "<IP_ADDRESS>",
<     "podIPs": [
<       {
<         "ip": "<IP_ADDRESS>"
<       }
<     ],
242,243c236,237
<           "running": {
<             "startedAt": "2022-07-12T23:55:35Z"
---
>           "waiting": {
>             "reason": "ContainerCreating"
246,247c240,249
<         "lastState": {},
<         "ready": true,
---
>         "lastState": {
>           "terminated": {
>             "exitCode": 137,
>             "reason": "ContainerStatusUnknown",
>             "message": "The container could not be located when the pod was deleted.  The container used to be Running",
>             "startedAt": null,
>             "finishedAt": null
>           }
>         },
>         "ready": false,
249,252c251,253
<         "image": "nginx:latest",
<         "imageID": "docker-pullable://nginx@sha256:dbe677093f569cc0afe2a149c529645f255aac959490ef11fb19ac6418b815d3",
<         "containerID": "docker://c547faec934393e8ec2e9450f8c49ae53ea75bf45bb763aa54594d9fd49887b5",
<         "started": true
---
>         "image": "nginx",
>         "imageID": "",
>         "started": false

UPDATE
6c6
<     "resourceVersion": "28873",
---
>     "resourceVersion": "28919",
8,9c8,9
<     "deletionTimestamp": "2022-07-12T23:56:13Z",
<     "deletionGracePeriodSeconds": 30,
---
>     "deletionTimestamp": "2022-07-12T23:55:43Z",
>     "deletionGracePeriodSeconds": 0,

REMOVE EVENT
This ends up making its way to Central because we are modifying running containers which we track. This will reduce the number of update events to pods
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
CI and metrics check

@fredrb Malte made a point about k8s finalizers which are fair, we should think about how we can throttle these


BUG: PyArray_MAX no longer available in NumPy

There was a change in NumPy which removes PyArray_MAX and PyArray_MIN. E.g. of issue in our CI https://github.com/scipy/scipy/actions/runs/5663008932/job/15343987040#step:9:2979
../scipy/integrate/_odepackmodule.c:477:12: error: implicit declaration of function ‘PyArray_MAX’; did you mean ‘PyArray_Max’? [-Werror=implicit-function-declaration]
  477 |     *lrw = PyArray_MAX(lrn,lrs);
      |            ^~~~~~~~~~~
      |            PyArray_Max

I believe the PR on NumPy is https://github.com/numpy/numpy/pull/24258

Can you comment on why this change and not importing PyArray_MAX from npy_math.h, where the linked numpy PR moves them to?

I have no clue what's best to do 😅 I just followed the bug, C and NumPy's internal are not my expertise. I am happy to try to do the change you are suggesting instead. And seems like CI does not like what I did hum.
So I would need to do #include "numpy/npy_math.h" in both files??

But if we use npy_math.h then it's not backward compatible no?

These includes might need to be guarded by an #ifdef depending on the numpy version? Let's ask @lysnikolaou for a recommended way of working around the numpy change.

Or maybe it's easiest to just copy-paste the macros in their full complexity and not depend on numpy for them:
#define PyArray_MAX(a,b) (((a)>(b))?(a):(b))
#define PyArray_MIN(a,b) (((a)<(b))?(a):(b))

Let's ask yes 👍
For now locally I got it working with:
out = (PyArrayObject*)PyArray_Max(a, b, NULL);
out = (PyArrayObject*)PyArray_Min(a, b, NULL);

I'm not an expert really, and with my limited understanding, I don't get why including "numpy/npy_math.h" is backwards-incompatible. If it is, then you can hide it behind an #ifdef, since the move will only be realeased in numpy 2.0.
Agreed that it might be best to copy-paste the macros though since they're not overly complex.

I don't get why including "numpy/npy_math.h" is backwards-incompatible.

We need to support NumPy >=1.22.4. So I would suppose that new changes in npy_math.h would not be backported/available on older versions. But I don't know how all that works.
I am happy to move the macros as there are just 2 files where these are used in our code base.

So I would suppose that new changes in npy_math.h would not be backported/available on older versions.

They won't be backported but my feeling is that if you keep both the old #include and the new #include "numpy/npy_math.h" in, it should be okay on all of the supported versions.
Still better to just move the macros.

Ah yes i see what you both meant now! See my C level 😅
I will update my PR to just hard code the macro there then, thanks again!


Fixes #736

Fixes issue where a hits file being written on process exit prevents the hits file from being read to generate coverage, as reported in #736 and discussed in #754

We're next release so we don't merge this now, I've got a similar simpler fix ready on my branch https://github.com/tonerdo/coverlet/compare/master...MarcoRossignoli:fix_736, but I need to understand how to fix that test(we expect correct counting on .NET Framework)

We're next release so we don't merge this now, I've got a similar simpler fix ready on my branch master...MarcoRossignoli:fix_736, but I need to understand how to fix that test(we expect correct counting on .NET Framework)

I'll close this then. Sorry, I thought this would make your life easier 😄

You support have been fundamental!
You did enough thanks!


Edit xml Coordinate reading loop

Was having issues reading "xml-splines", especially when numbers of parametric & physical dimensions didn't match. This seems to solve the issue!

Thanks for pointing this out! I applied the fix to the initial commit :)


Warning during compilation from source (ytpapi2, unused import)

After having the latest release not working for me (i had this issue), I decided to build the binary manually.  I had an unused import warning for ytpapi2, and the logfile reports this :

INFO - Logger mode INFO [src/systems/logger.rs]
INFO - Init: 0ms [src/structures/performance.rs]
INFO - Startup: 0ms [src/structures/performance.rs]
INFO - Spawned clean task: 0ms [src/structures/performance.rs]
INFO - Clean task: 0ms [src/structures/performance.rs]
INFO - Spawned system task: 41ms [src/structures/performance.rs]
INFO - Spawned last playlist task: 41ms [src/structures/performance.rs]
INFO - Spawned api task: 41ms [src/structures/performance.rs]
INFO - Running manager: 41ms [src/structures/performance.rs]
INFO - Database getter task on [src/tasks/local_musics.rs]
INFO - Local musics: 0ms [src/structures/performance.rs]
INFO - Last playlist task on [src/tasks/last_playlist.rs]
INFO - API task on [src/tasks/api.rs]
INFO - Last playlist: 0ms [src/structures/performance.rs]
ERROR - Error in browse [ytpapi2/src/lib.rs]
ERROR - Object({"error": Object({"code": Number(401), "errors": Array([Object({"domain": String("global"), "message": String("You must be signed in to perform this operation."), "reason": String("unauthorized")})]), "message": String("You must be signed in to perform this operation."), "status": String("UNAUTHENTICATED")})}) [ytpapi2/src/lib.rs]
ERROR - YoutubeMusicError(Object({"error": Object({"code": Number(401), "errors": Array([Object({"domain": String("global"), "message": String("You must be signed in to perform this operation."), "reason": String("unauthorized")})]), "message": String("You must be signed in to perform this operation."), "status": String("UNAUTHENTICATED")})})) [src/tasks/api.rs]
ERROR - Error in browse [ytpapi2/src/lib.rs]
ERROR - Object({"error": Object({"code": Number(401), "errors": Array([Object({"domain": String("global"), "message": String("You must be signed in to perform this operation."), "reason": String("unauthorized")})]), "message": String("You must be signed in to perform this operation."), "status": String("UNAUTHENTICATED")})}) [ytpapi2/src/lib.rs]
ERROR - YoutubeMusicError(Object({"error": Object({"code": Number(401), "errors": Array([Object({"domain": String("global"), "message": String("You must be signed in to perform this operation."), "reason": String("unauthorized")})]), "message": String("You must be signed in to perform this operation."), "status": String("UNAUTHENTICATED")})})) [src/tasks/api.rs]
INFO - API task: 1712ms [src/structures/performance.rs]
INFO - Browse playlist Louise Attaque: 436ms [src/structures/performance.rs]
INFO - Browse playlist Du soleil au coeur: 442ms [src/structures/performance.rs]
INFO - Browse playlist Dans la légende: 452ms [src/structures/performance.rs]
INFO - Browse playlist MESDAMES deluxe: 470ms [src/structures/performance.rs]
INFO - Browse playlist NAKAMURA: 492ms [src/structures/performance.rs]
INFO - Browse playlist Destin: 498ms [src/structures/performance.rs]
INFO - Browse playlist Inspi d'ailleurs: 503ms [src/structures/performance.rs]
INFO - Browse playlist Mélo: 506ms [src/structures/performance.rs]
INFO - Browse playlist Kendji: 510ms [src/structures/performance.rs]
INFO - Browse playlist Starboy: 532ms [src/structures/performance.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading 8i0rXZcEyIA: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading FSpkIHuYn2E: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading eCxjpGhorPs: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading Op_UhnD6CmE: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading -IMM6OlbDIc: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading sK4_drvjb4U: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading wgFDXSCHkgo: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading 15qFDdYTUFI: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading Gp7TfLKUXxE: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading o3Dv4L2RnRQ: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading fmQ6t8evUCU: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading NqxwOURlvdQ: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading 6nALmZ1cjwE: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading xDE-YIHMgdM: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading Hzu03Y7BtCY: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading zbeuxm8-Inc: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading ZHoxCiU4Lj8: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading 3rKCUq5PCqY: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading W2M_7Et8o2c: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]
ERROR - fetch() => UnexpectedResponse("watch html did not contain a PlayabilityStatus") [/home/lae/.cargo/git/checkouts/rustube-48cbb966bdb4b8ab/3b1e82f/src/fetcher.rs]
ERROR - Error downloading GdGKlTyGeR0: YouTube returned an unexpected response: `watch html did not contain a PlayabilityStatus` [src/tasks/download.rs]```

This most likely means your headers are invalid... The unused import is fixed on beta branch waiting for some reviews before merging.


Enable Probot

This closes #141

Be aware, @mienaikage that the label will be "wontfix" and perhaps should be "stale" instead.  Up to you.  See exercism.io/.github/stale.yml.

Stale seems to make more sense than wontfix.


Initial faction storage implementation (+ enabled kotlin support)

The current implementation works by marking the coordinates of ender chests are to be used for faction inventories. This is checked through an inject mixin on EnderChestBlock.onUse.
Individual chests are marked using the /f toggleChestMode <X> <Y> <Z> command.
Pending proper testing, hence the draft status of the PR.
This is an alternative to #4 .

Initial test results: the feature is confirmed to be working properly.
Known bugs:

Closing animation fails to trigger for faction chests. (Exclusively visual bug)
Clients running fabric (with certain mods) error out when connecting to server.
(seems to be an issue with the command registry)

Cause of the issue regarding connecting to the faction servers for fabric clients has been identified.
The cause was a version incompatebility. If both client/server use the latest fabric loader/api version then it works properly


Route entries get deleted by felix due to stale data from typha

Expected Behavior

In a kubernetes cluster using calico (v3.21.1), pod network works so that pods can communicate with each other and the outside world according to network policies.
Current Behavior

On some nodes the communication from within a pod stops working since the route from node to pod gets removed. In this case for <IP_ADDRESS>/32. The interface (cali1037a54e65e) however still exists.
2021-12-10 14:10:42.201 [INFO][22936] felix/summary.go 100: Summarising 11 dataplane reconciliation loops over 1m3.3s: avg=37ms longest=316ms ()
2021-12-10 14:10:51.442 [INFO][106] monitor-addresses/startup.go 741: Using autodetected IPv4 address <IP_ADDRESS>/22 on interface ens192 matching cidrs [<IP_ADDRESS>/22]
2021-12-10 14:11:31.653 [INFO][22936] felix/int_dataplane.go 1071: Linux interface addrs changed. addrs=set.mapSet{} ifaceName="cali1037a54e65e"
2021-12-10 14:11:31.653 [INFO][22936] felix/int_dataplane.go 1036: Linux interface state changed. ifIndex=751 ifaceName="cali1037a54e65e" state="up"
2021-12-10 14:11:31.653 [INFO][22936] felix/int_dataplane.go 1539: Received interface addresses update msg=&intdataplane.ifaceAddrsUpdate{Name:"cali1037a54e65e", Addrs:set.mapSet{}}
2021-12-10 14:11:31.653 [INFO][22936] felix/hostip_mgr.go 85: Interface addrs changed. update=&intdataplane.ifaceAddrsUpdate{Name:"cali1037a54e65e", Addrs:set.mapSet{}}
2021-12-10 14:11:31.654 [INFO][22936] felix/int_dataplane.go 1521: Received interface update msg=&intdataplane.ifaceUpdate{Name:"cali1037a54e65e", State:"up", Index:751}
2021-12-10 14:11:31.654 [INFO][22936] felix/endpoint_mgr.go 366: Workload interface came up, marking for reconfiguration. ifaceName="cali1037a54e65e"
2021-12-10 14:11:31.654 [INFO][22936] felix/endpoint_mgr.go 1179: Applying /proc/sys configuration to interface. ifaceName="cali1037a54e65e"
2021-12-10 14:11:31.655 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:31.656 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:31.656 [INFO][22936] felix/route_table.go 567: Interface in cleanup grace period, will retry after. ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:33.095 [INFO][22936] felix/iface_monitor.go 201: Netlink address update. addr="fe80::ecee:eeff:feee:eeee" exists=true ifIndex=751
2021-12-10 14:11:33.095 [INFO][22936] felix/int_dataplane.go 1071: Linux interface addrs changed. addrs=set.mapSet{"fe80::ecee:eeff:feee:eeee":set.empty{}} ifaceName="cali1037a54e65e"
2021-12-10 14:11:33.095 [INFO][22936] felix/int_dataplane.go 1539: Received interface addresses update msg=&intdataplane.ifaceAddrsUpdate{Name:"cali1037a54e65e", Addrs:set.mapSet{"fe80::ecee:eeff:feee:eeee":set.empty{}}}
2021-12-10 14:11:33.095 [INFO][22936] felix/hostip_mgr.go 85: Interface addrs changed. update=&intdataplane.ifaceAddrsUpdate{Name:"cali1037a54e65e", Addrs:set.mapSet{"fe80::ecee:eeff:feee:eeee":set.empty{}}}
2021-12-10 14:11:33.096 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:33.097 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:33.097 [INFO][22936] felix/route_table.go 567: Interface in cleanup grace period, will retry after. ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:35.029 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:35.030 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:35.031 [INFO][22936] felix/route_table.go 567: Interface in cleanup grace period, will retry after. ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:39.125 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:39.126 [INFO][22936] felix/route_table.go 892: Syncing routes: found unexpected route; ignoring due to grace period. dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:39.126 [INFO][22936] felix/route_table.go 567: Interface in cleanup grace period, will retry after. ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2021-12-10 14:11:45.779 [INFO][22936] felix/route_table.go 896: Remove old route dest=<IP_ADDRESS>/32 ifaceName="cali1037a54e65e" ifaceRegex="^cali.*" ipVersion=0x4 routeProblems=[]string{"unexpected route"} tableIndex=0
2021-12-10 14:11:45.779 [INFO][22936] felix/conntrack.go 90: Removing conntrack flows ip=<IP_ADDRESS>
2021-12-10 14:11:45.782 [INFO][22936] felix/summary.go 100: Summarising 13 dataplane reconciliation loops over 1m3.6s: avg=72ms longest=703ms ()
2021-12-10 14:11:51.465 [INFO][106] monitor-addresses/startup.go 741: Using autodetected IPv4 address <IP_ADDRESS>/22 on interface ens192 matching cidrs [<IP_ADDRESS>/22]

As consequence of the deleted routes we saw in the kernel ring buffer the following martian packets(data from another cluster):
[3643049.978035] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643049.984308] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06
[3643052.927980] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643052.934388] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06
[3643053.946057] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643053.952332] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06
[3643054.970021] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643054.976333] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06
[3643057.928153] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643057.934515] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06
[3643058.937977] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643058.944272] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06
[3643059.961983] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cali305be8838b5
[3643060.062278] ll header: 00000000: ff ff ff ff ff ff 3a cd 6b 6b 10 ff 08 06

Some of the existing pods on a failing node are effected. New pods on a failing node are always effected.
Possible Solution

Steps to Reproduce (for bugs)

Context

We use typha with multiple instances, in this case two. All calico-nodes with issues were connected to one typha instance while the rest of the cluster seems to work as expected. A restart of the "stale" typha resolved the problems.
Reconfiguring felix through a change to the felixconfiguration seems to trigger a reconnect to typha. Using this we could toggle between the "stale" and working typha instance.
The issue was first observed after updating to calico v3.21.x, v3.19.x and v3.20.x work as expected.
Your Environment

Calico version v3.21.1
Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes
Operating System and version:

/cc: @scheererj

This kind of issue is typically caused by a bug in the kubernetes client.  v3.20.x used kube client v0.21.0-rc.0 but v3.21 uses v0.21.0.  Perhaps there was a regression.

@DockToFuture Thanks for raising this. We would like to gather additional information. Specifically, can you confirm:

the current K8s version and any other versions you have observed this on
details on the K8s environment - particularly the build process and anything unusual or custom, such as extra networking hops, load balancers, or anything like that.

The k8s versions we observed the problem on were 1.19.15, 1.20.9 and 1.22.2.
We use https://github.com/gardener/gardener for creating and managing our k8s clusters.
The underlying infrastructures were openstack, aws and azure.
The controlplane is in a different infrastructure account than the dataplane, see https://gardener.cloud for more details. Between the dataplane and the kube-apiserver is a public loadbalancer and an istio ingress gateway. The traffic going via the kubernetes.default.svc.cluster.local service goes over an additional envoy proxy.

Eek, that's a lot of moving parts between Calico and your API server.  Calico relies on HTTP2 keepalives to make sure the connection to the API server is up.  I wonder if one of your intermediate proxies is preventing keepalives from flowing end-to-end.

This setup has worked for more than a year like that with different calico versions. Only with calico v3.21.x we sometimes see the issue described here.

I've tried to reproduce the issue by simulating a silent failure of the API server connection but in my tests, the failure is detected as expected.  Are you sure that only Calico changed to cause the issue?  Do different versions of Gardener also come with different versions of Envoy/Istio/etc?

Currently we use the same istio and envoy version for all clusters and it works smoothly with the old/prior calico versions.

AKS started deploying v3.21.0 today and on each clusters where the new version was deployed, we started seeing the same issue, routes were removed, we even saw Felix trying to reach out to typha pods that didn't exists anymore. (wrong IP).  Due to the impact it had, it was reverted back to v3.20.0 and it's now stable, we didn't keep around the calico-node logs. I would try to dig in our logging backend to see if we are indexing the calico-node logs.

I'd like to look into this, but probably need a full log file from a calico/typha + calico/node that are experiencing this issue, and if possible a /var/log/calico/cni/cni.log from the node as well (assuming it is using calico CNI).
This is the full set of changes from v3.20.0 -> v3.21.2 in the calico/typha codebase: https://github.com/projectcalico/typha/compare/v3.20.0...v3.21.2
There's not much there, but a few things stand out as potential disruptions:

https://github.com/projectcalico/typha/pull/614/
https://github.com/projectcalico/typha/pull/694
https://github.com/projectcalico/typha/pull/671

It may have also been a change in libcalico-go though, as much of connection logic to the apiserver lives there.
These are the changes in libcalico between typha v3.20.0 and v3.21.2: https://github.com/projectcalico/libcalico-go/compare/279457908a6...3402eca9d27
The main changes that stand out to me are:

https://github.com/projectcalico/libcalico-go/pull/1484

Some other thoughts:

AKS started deploying v3.21.0 today

v3.21.0 had a bug fixed in v3.21.1 which impacted pods that made use of Kubernetes hostPorts - if the pods that were losing connectivity all had hostPorts (specifically without having a name set) then it is likely you were hitting the issue fixed here: https://github.com/projectcalico/libcalico-go/pull/1545

Felix gets this information by direct query of the Kubernetes Endpoints API. Maybe this is the apiserver telling Felix the wrong information? Or Maybe the Typha pod goes away and Felix never realizes it, and so never re-queries the apiserver for a new endpoint?

Casey is right, when Felix starts up, it queries the API server directly for the list of service endpoints
The Felix/typha prototcol includes a layer-7 heartbeat.  If Felix doesn't hear from typha for 30s then it will disconnect and restart.  Then it goes through discovery again, reading the list of typha endpoints again and choosing a fresh one.

@DockToFuture @djsly When you say that you're rolling out Calico v3.21.x, is that the only change you're making on these clusters?  Can you give us a timeline of events?  You apply the new v3.21.x update and then how long until you notice stale routes?

Hello @fasaxc, once we rolled it out it took a while until we noticed the first issues with that problem. We saw it sporadically on a few clusters but not on the majority of clusters.
We reverted our calico version back to the previous one prior v3.21.x where we faced no issues.

Thanks @DockToFuture, are we talking 1-in-100 clusters, 1-in-10?  By "while" are we talking 15 mins, or, you upgraded one day and then found issues a day later (for example)?
Anything that stands out about the affected clusters?  Were they small clusters; large clusters; high churn rate of pods (or the opposite)?

We are in the range of ~10 out of 1000 clusters where we noticed it. We saw the issue on small and on large clusters. I would say it was evenly distributed. We noticed the issue after a couple of days, but since we also using a hibernation feature on many clusters, the issue gets resolved on its own due to a restart of typha. So there is a lot of uncertainty in the observations. I can not tell if the issue occurred directly after the rollout of the new calico version or if it took some time for typha to get in that "stale" state.

for me, AKS started the upgrade in the background, we saw the issue on 4 / 4 of the upgraded clusters, It seems that for the first cluster, nothing was failing for 24hours until we started deploying new pods (scalling up a deployments). So post calico upgrade, nothing seems to fail right away, but once we started performing operations, thats when users started reporting issues.
For us, it was ALL clusters that were affected post calico upgrades, we are running AKS 1.20.9 with both linux and windows node pools , one cluster has 50 nodes with 1000 pods while the other has  over 400 nodes and ~7500 pods

OK, so that is consistent with typha starting, connecting to the datastore, downloading a snapshot and generally working for a while.  Then, at some point, the connection to the datastore stalls and typha gets stuck.  Since typha is a caching proxy, it'll cache the most recent state of the datastore that it has seen and that's what felix will see if it connects to that typha.

We're struggling to reproduce this issue.  We've made cluster with hundreds of typha instances and using their prometheus metrics to look for any laggards but so far no luck.
@DockToFuture @ffilippopoulos  would you be able to try test builds of typha with extra diagnostics and some of the candidate causes backed out?

have you tried putting extra stress on the apiserver ? causing watch cache connections to drop / die/ reconnect.  that would be my guess

one of the cluster, the issue started ~2h after, when the deployments started, let me try to find some time today to extra the logs, you wanted Typha and Calico Node ?

Yes please, and any timeline information about when exactly you saw what problems would be useful.

here are logs from 3:30 UTM to 4:30 UTM, users reported seeing login failure at 4:00UTC
logs are from ALL calico pods (typha/node(s))
query_data-5.csv

Logs standing out
2021-12-17 15:27:07.984 [INFO][6] watchercache.go 188: Failed to perform list of current data during resync ListRoot="/calico/resources/v3/projectcalico.org/workloadendpoints" error=ListOptions.meta.k8s.io "" is invalid: [resourceVersionMatch: Forbidden: resourceVersionMatch is forbidden unless resourceVersion is provided, resourceVersionMatch: Forbidden: resourceVersionMatch is forbidden when continue is provided]

Hmm, I think that might be a bug in the Kubernetes client, based on a quick investigation.
That error is coming from here: https://github.com/kubernetes/apimachinery/blob/master/pkg/apis/meta/internalversion/validation/validation.go#L35-L37
The Calico code uses the client-go list pager to perform paged List() calls against the Kubernetes API here, passing in resourceVersion as well as resourceVersionMatch as needed: https://github.com/projectcalico/calico/blob/8fdb4215535b71117917bfa8871abe2652ec3a25/libcalico-go/lib/backend/k8s/resources/list_pager.go#L43-L50
The client-go code properly clears resourceVersion on page continuations, but doens't clear resourceVersionMatch: https://github.com/kubernetes/client-go/blob/master/tools/pager/pager.go#L138-L141
I wonder if that is intentional, if so I don't see a way for us to set resourceVersionMatch on paged list calls, so I would guess it is  not.

I've raised the following issue to see what the client-go folks say: https://github.com/kubernetes/client-go/issues/1042

would you be able to try test builds of typha with extra diagnostics and some of the candidate causes backed out?

sure, we could try this on lower environments here, and keep an eye. Even though we have only seen the issue surfacing on our prod cluster so far for some reason.

I've built a test image with @caseydavenport 's suggested patch to the Kubernetes client library and pushed as docker image: fasaxc/typha:v3.21-with-pager-fix  This should be a drop-in replacement for calico/typha:v3.21.x (on AMD64 only) if you want to try it.  I was able to write a unit test that showed the above error before applying the fix and the error cleared when I applied the fix.
I'm pretty sure the fix is correct; the reason for blocking the revision-related parameters when doing a paged read is that the pagination information is stored server side and it wouldn't make sense to change the revision on subsequent paged reads.

Hi, we also had some issues when upgrading to v3.21.1 and had to revert to v3.20 which after some investigating it turned out to be caused by the same issue described here.
It seems the fix didn't make it to v3.21.2 and were wondering if it will be included in v3.21.3 release.

Yes, it'll be in v3.21.3; it was backported here: https://github.com/projectcalico/calico/pull/5332.


Multi-instance docker-compose

Multi-instance docker-compose, for physical servers with multiple NICs or IPs
Network:host has bug,now work all good

...... I don't think we need to add this docker compose configuration.
For docker compose, the default port configuration will be bound to all NICs;
ports:
   - '36711:36711/udp'

If you have multiple IPs and expect each IP to use a different port, you only need to modify the port configuration:
ports:
   - '<IP_ADDRESS>:36711:36711/udp'
   - '<IP_ADDRESS>:36712:36711/udp'

For the port configuration of docker compose, you can refer to the official document https://docs.docker.com/compose/compose-file/compose-file-v3/#short-syntax-1
By the way, the following port configurations are all legal:
ports:
  - "3000"
  - "3000-3005"
  - "8000:8000"
  - "9090-9091:8080-8081"
  - "49100:22"
  - "<IP_ADDRESS>:8001:8001"
  - "<IP_ADDRESS>:5000-5010:5000-5010"
  - "<IP_ADDRESS>::5000"
  - "6060:6060/udp"
  - "12400-12500:1240"


Documentation: on precision

I couldn't figure out the function of the --precision flag. According to the page here: https://trafilatura.readthedocs.io/en/latest/evaluation.html, the standard version is more accurate while the precision variant is more precise?
What does that mean in real practical terms?
It would have been great if the documentation can clarify how the --precision flag changes the output; showcasing with  practical examples.

You can also have a look at the code to see which additional parts are discarded, and you can run the benchmark in the tests/ folder to see how extraction options compare to the rest.


Incorrect interpretation of '' in Bicep parameter.

Bicep version
az bicep version
Bicep CLI version 0.4.1318 (ee0d808f35)
Describe the bug
A clear and concise description of what the bug is vs what you expected to happen
Linter shows warning when used '' in if-else condition for evaluation. Linter should interprete '' as null and not show warning.
To Reproduce
Steps to reproduce the behavior:
@allowed([
'AAD'
'PrivateLink'
''
])
param allowedCopyScope string = ''
When used with the if-else condition in resource or variable, linter shows following warning. see screenshot -
allowedCopyScope: ((allowedCopyScope == '' ) ? null : allowedCopyScope)

allowedCopyScope: ((allowedCopyScope == null ) ? null : allowedCopyScope)

Can you share a more complete code sample, including the full resource declaration?
It seems like Bicep is saying that empty string '' is not a valid value for this particular property. What happens if you try to directly set the value of this property to '' without a parameter?

Hi @alex-frankel
Below is the complete code sample I wrote for Storage Deployment.
`
@description('Required. The name of the storage account within the specified resource group. Storage account names must be between 3 and 24 characters in length and use numbers and lower-case letters only.')
@minLength(3)
@maxLength(24)
param name string = uniqueString(resourceGroup().id)
@description('The location of the resource. This will be one of the supported and registered Azure Geo Regions (e.g. West US, East US, Southeast Asia, etc.). The geo region of a resource cannot be changed once it is created, but if an identical geo region is specified on update, the request will succeed.')
param location string = resourceGroup().location
@description('Sets a list of key value pairs that describe the resource. These tags can be used for viewing and grouping this resource (across resource groups). A maximum of 15 tags can be provided for a resource. Each tag must have a key with a length no greater than 128 characters and a value with a length no greater than 256 characters.')
param tags object = {}
@description('The SKU name. Required for account creation; optional for update. Note that in older versions, SKU name was called accountType.Default value is Standard_LRS.')
@allowed([
'Premium_LRS'
'Premium_ZRS'
'Standard_GRS'
'Standard_GZRS'
'Standard_LRS'
'Standard_RAGRS'
'Standard_RAGZRS'
'Standard_ZRS'
])
param sku_Name string = 'Standard_LRS'
@description('Indicates the type of storage account.')
@allowed([
'BlobStorage'
'BlockBlobStorage'
'FileStorage'
'Storage'
'StorageV2'
])
param kind string = 'StorageV2'
@description('The name of the extended location.')
param extendedLocation_Name string = ''
@description('Default value: None. The type of identity used for the resource. The type "SystemAssigned,UserAssigned" includes both an implicitly created identity and a set of user assigned identities. The type "None" will remove any identities from the service.')
@allowed([
'None'
'SystemAssigned'
'SystemAssigned,UserAssigned'
'UserAssigned'
])
param identity_Type string = 'None'
@description('Sets a list of key value pairs that describe the set of User Assigned identities that will be used with this storage account. The key is the ARM resource identifier of the identity. Only 1 User Assigned identity is permitted here.')
param identity_UserAssignedIdentities object = {}
@description('Optional. Required for storage accounts where kind = BlobStorage. The access tier used for billing. - Hot or Cool. Default value is Hot.')
@allowed([
'Hot'
'Cool'
])
param accessTier string = 'Hot'
@description('Allow or disallow public access to all blobs or containers in the storage account. The default interpretation is true for this property.')
param allowBlobPublicAccess bool = true
@description('Allow or disallow cross AAD tenant object replication. The default interpretation is true for this property.')
param allowCrossTenantReplication bool = true
@description('Restrict copy to and from Storage Accounts within an AAD tenant or with Private Links to the same VNet.')
@allowed([
'AAD'
'PrivateLink'
''
])
param allowedCopyScope string = ''
@description('Indicates whether the storage account permits requests to be authorized with the account access key via Shared Key. If false, then all requests, including shared access signatures, must be authorized with Azure Active Directory (Azure AD). The default value is null, which is equivalent to true.')
param allowSharedKeyAccess bool = true
@description('Specifies the Active Directory account type for Azure Storage.')
@allowed([
'Computer'
'User'
''
])
param azFilesIBA_ADProperties_AccountType string = ''
@description('Specifies the security identifier (SID) for Azure Storage.')
param azFilesIBA_ADProperties_AzStorageSid string = ''
@description('Specifies the domain GUID.')
param azFilesIBA_ADProperties_DomainGuid string = ''
@description('Specifies the primary domain that the AD DNS server is authoritative for.')
param azFilesIBA_ADProperties_DomainName string = ''
@description('Specifies the security identifier (SID).')
param azFilesIBA_ADProperties_DomainSid string = ''
@description('Specifies the Active Directory forest to get.')
param azFilesIBA_ADProperties_ForestName string = ''
@description('Specifies the NetBIOS domain name.')
param azFilesIBA_ADProperties_NetBiosDomainName string = ''
@description('Specifies the Active Directory SAMAccountName for Azure Storage.')
param azFilesIBA_ADProperties_SamAccountName string = ''
@description('Default share permission for users using Kerberos authentication if RBAC role is not assigned.')
@allowed([
'None'
'StorageFileDataSmbShareContributor'
'StorageFileDataSmbShareElevatedContributor'
'StorageFileDataSmbShareReader'
])
param azFilesIBA_DefaultSharePermission string = 'None'
@description('Indicates the directory service used.')
@allowed([
'AADDS'
'AD'
'None'
])
param azFilesIBA_DirectoryServiceOptions string = 'None'
@description('Gets or sets the custom domain name assigned to the storage account. Name is the CNAME source.')
param customDomain_Name string = ''
@description('Indicates whether indirect CName validation is enabled. Default value is false. This should only be set on updates.')
param customDomain_UseSubDomainName bool = false
@description('A boolean flag which indicates whether the default authentication is OAuth or not. The default interpretation is false for this property.')
param defaultToOAuthAuthentication bool = false
@description('ClientId of the multi-tenant application to be used in conjunction with the user-assigned identity for cross-tenant customer-managed-keys server-side encryption on the storage account')
param encryption_Identity_FederatedIdentityClientId string = ''
@description('Resource identifier of the UserAssigned identity to be associated with server-side encryption on the storage account.')
param encryption_Identity_UserAssignedIdentity string = ''
@description('The encryption keySource (provider). Possible values (case-insensitive): Microsoft.Storage, Microsoft.Keyvault')
@allowed([
'Microsoft.Storage'
'Microsoft.Keyvault'
])
param encryption_KeySource string = 'Microsoft.Storage'
@description('The name of KeyVault key.')
param encryption_KeyVaultProperties_KeyName string = ''
@description('The Uri of KeyVault.')
param encryption_KeyVaultProperties_KeyVaultUri string = ''
@description('The version of KeyVault key.')
param encryption_KeyVaultProperties_KeyVersion string = ''
@description('A boolean indicating whether or not the service applies a secondary layer of encryption with platform managed keys for data at rest.')
param encryption_RequireInfrastructureEncryption bool = false
@description('A boolean indicating whether or not the service encrypts the data as it is stored. Encryption at rest is enabled by default today and cannot be disabled.')
param encryption_Services_Blob_Enabled bool = true
@description('Encryption key type to be used for the encryption service. 'Account' key type implies that an account-scoped encryption key will be used. 'Service' key type implies that a default service key is used.')
@allowed([
'Account'
'Service'
])
param encryption_Services_Blob_KeyType string = 'Account'
@description('A boolean indicating whether or not the service encrypts the data as it is stored. Encryption at rest is enabled by default today and cannot be disabled.')
param encryption_Services_File_Enabled bool = true
@description('Encryption key type to be used for the encryption service. 'Account' key type implies that an account-scoped encryption key will be used. 'Service' key type implies that a default service key is used.')
@allowed([
'Account'
'Service'
])
param encryption_Services_File_KeyType string = 'Account'
@description('A boolean indicating whether or not the service encrypts the data as it is stored. Encryption at rest is enabled by default today and cannot be disabled.')
param encryption_Services_Queue_Enabled bool = true
@description('Encryption key type to be used for the encryption service. 'Account' key type implies that an account-scoped encryption key will be used. 'Service' key type implies that a default service key is used.')
@allowed([
'Account'
'Service'
])
param encryption_Services_Queue_KeyType string = 'Account'
@description('A boolean indicating whether or not the service encrypts the data as it is stored. Encryption at rest is enabled by default today and cannot be disabled.')
param encryption_Services_Table_Enabled bool = true
@description('Encryption key type to be used for the encryption service. 'Account' key type implies that an account-scoped encryption key will be used. 'Service' key type implies that a default service key is used.')
@allowed([
'Account'
'Service'
])
param encryption_Services_Table_KeyType string = 'Account'
@description('A boolean flag which enables account-level immutability. All the containers under such an account have object-level immutability enabled by default.')
param immutableStorageWithVersioning_Enabled bool = false
@description('This property can only be changed for disabled and unlocked time-based retention policies. When enabled, new blocks can be written to an append blob while maintaining immutability protection and compliance. Only new blocks can be added and any existing blocks cannot be modified or deleted.')
param immutableStorageWithVersioning_ImmutabilityPolicy_AllowProtectedAppendWrites bool = false
@description('The immutability period for the blobs in the container since the policy creation, in days.')
param immutableStorageWithVersioning_ImmutabilityPolicy_ImmutabilityPeriodSinceCreationInDays int = 0
@description('The ImmutabilityPolicy state defines the mode of the policy. Disabled state disables the policy, Unlocked state allows increase and decrease of immutability retention time and also allows toggling allowProtectedAppendWrites property, Locked state only allows the increase of the immutability retention time. A policy can only be created in a Disabled or Unlocked state and can be toggled between the two states. Only a policy in an Unlocked state can transition to a Locked state which cannot be reverted.')
@allowed([
'Disabled'
'Locked'
'Unlocked'
''
])
param immutableStorageWithVersioning_ImmutabilityPolicy_State string = ''
@description('Account HierarchicalNamespace enabled if sets to true.')
param isHnsEnabled bool = false
@description('Enables local users feature, if set to true.')
param isLocalUserEnabled bool = false
@description('NFS 3.0 protocol support enabled if set to true.')
param isNfsV3Enabled bool = false
@description('Enables Secure File Transfer Protocol, if set to true.')
param isSftpEnabled bool = false
@description('The key expiration period in days.')
param keyPolicy_KeyExpirationPeriodInDays int = 0
@description('Allow large file shares if sets to Enabled. It cannot be disabled once it is enabled.')
@allowed([
'Disabled'
'Enabled'
])
param largeFileSharesState string = 'Disabled'
@description('description')
@allowed([
'TLS1_0'
'TLS1_1'
'TLS1_2'
])
param minimumTlsVersion string = 'TLS1_2'
@description('Specifies whether traffic is bypassed for Logging/Metrics/AzureServices. Possible values are any combination of Logging|Metrics|AzureServices (For example, 'Logging, Metrics'), or None to bypass none of those traffics.')
@allowed([
'Logging'
'Logging, Metrics'
'Logging, AzureServices'
'Logging, Metrics, AzureServices'
'Metrics'
'Metrics, AzureServices'
'AzureServices'
'None'
])
param networkAcls_Bypass string = 'None'
@description('Specifies the default action of allow or deny when no other rules match.')
@allowed([
'Allow'
'Deny'
])
param networkAcls_DefaultAction string = 'Deny'
@description('Sets the IP ACL rules.')
param networkAcls_IpRules array = []
@description('Sets the resource access rules.')
param networkAcls_ResourceAccessRules array = []
@description('Sets the virtual network rule.')
param networkAcls_VirtualNetworkRules array = []
@description('description')
@allowed([
'Enabled'
'Disabled'
])
param publicNetworkAccess string = 'Disabled'
@description('A boolean flag which indicates whether internet routing storage endpoints are to be published.')
param routingPreference_PublishInternetEndpoints bool = false
@description('A boolean flag which indicates whether microsoft routing storage endpoints are to be published.')
param routingPreference_PublishMicrosoftEndpoints bool = false
@description('Routing Choice defines the kind of network routing opted by the user.')
@allowed([
'InternetRouting'
'MicrosoftRouting'
])
param routingPreference_RoutingChoice string = 'MicrosoftRouting'
@description('The SAS expiration action. Can only be Log.')
@allowed([
'Log'
''
])
param sasPolicy_ExpirationAction string = ''
@description('The SAS expiration period, DD.HH:MM:SS.')
param sasPolicy_SasExpirationPeriod string = ''
@description('description')
param supportsHttpsTrafficOnly bool = true
var extendedLocation = {
name: extendedLocation_Name
type: 'EdgeZone'
}
var azFilesIdentityBasedAuthentication = {
azureFilesIdentityBasedAuthentication: {
activeDirectoryProperties: {
accountType: azFilesIBA_ADProperties_AccountType
azureStorageSid: azFilesIBA_ADProperties_AzStorageSid
domainGuid: azFilesIBA_ADProperties_DomainGuid
domainName: azFilesIBA_ADProperties_DomainName
domainSid: azFilesIBA_ADProperties_DomainSid
forestName: azFilesIBA_ADProperties_ForestName
netBiosDomainName: azFilesIBA_ADProperties_NetBiosDomainName
samAccountName: azFilesIBA_ADProperties_SamAccountName
defaultSharePermission: azFilesIBA_DefaultSharePermission
directoryServiceOptions: azFilesIBA_DirectoryServiceOptions
}
}
}
var customDomain = {
name: customDomain_Name
useSubDomainName: customDomain_UseSubDomainName
}
var encryption_Identity = {
federatedIdentityClientId: encryption_Identity_FederatedIdentityClientId
userAssignedIdentity: encryption_Identity_UserAssignedIdentity
}
var encryption_KeyVaultProperties = {
keyname: encryption_KeyVaultProperties_KeyName
keyvaulturi: encryption_KeyVaultProperties_KeyVaultUri
keyversion: encryption_KeyVaultProperties_KeyVersion
}
var immutabilityPolicy = {
allowProtectedAppendWrites: immutableStorageWithVersioning_ImmutabilityPolicy_AllowProtectedAppendWrites
immutabilityPeriodSinceCreationInDays: immutableStorageWithVersioning_ImmutabilityPolicy_ImmutabilityPeriodSinceCreationInDays
state: immutableStorageWithVersioning_ImmutabilityPolicy_State
}
var keyPolicy = {
keyExpirationPeriodInDays: keyPolicy_KeyExpirationPeriodInDays
}
var sasPolicy = {
expirationAction: sasPolicy_ExpirationAction
sasExpirationPeriod: sasPolicy_SasExpirationPeriod
}
resource storageAccounts 'Microsoft.Storage/storageAccounts@2021-08-01' = {
name: replace(toLower(name), ' ', '')
location: location
tags: tags
sku: {
name: sku_Name
}
kind: kind
extendedLocation: ((extendedLocation_Name == '') ? null : extendedLocation)
identity: {
type: identity_Type
userAssignedIdentities: identity_UserAssignedIdentities
}
properties: {
accessTier: accessTier
allowBlobPublicAccess: allowBlobPublicAccess
allowCrossTenantReplication: allowCrossTenantReplication
allowedCopyScope: ((allowedCopyScope == '') ? null : allowedCopyScope)
allowSharedKeyAccess: allowSharedKeyAccess
azureFilesIdentityBasedAuthentication: ((azFilesIBA_ADProperties_AccountType == '') ? null : azFilesIdentityBasedAuthentication)
customDomain: ((customDomain_Name == '') ? null : customDomain)
defaultToOAuthAuthentication: defaultToOAuthAuthentication
encryption: {
identity: ((encryption_Identity_FederatedIdentityClientId == '') ? null : encryption_Identity)
keySource: encryption_KeySource
keyvaultproperties: ((encryption_KeySource == 'Microsoft.Keyvault') ? encryption_KeyVaultProperties : null)
requireInfrastructureEncryption: encryption_RequireInfrastructureEncryption
services: {
blob: {
enabled: encryption_Services_Blob_Enabled
keyType: encryption_Services_Blob_KeyType
}
file: {
enabled: encryption_Services_File_Enabled
keyType: encryption_Services_File_KeyType
}
queue: {
enabled: encryption_Services_Queue_Enabled
keyType: encryption_Services_Queue_KeyType
}
table: {
enabled: encryption_Services_Table_Enabled
keyType: encryption_Services_Table_KeyType
}
}
}
immutableStorageWithVersioning: {
enabled: immutableStorageWithVersioning_Enabled
immutabilityPolicy: ((immutableStorageWithVersioning_ImmutabilityPolicy_State == '') ? null : immutabilityPolicy)
}
isHnsEnabled: isHnsEnabled
isLocalUserEnabled: isLocalUserEnabled
isNfsV3Enabled: isNfsV3Enabled
isSftpEnabled: isSftpEnabled
keyPolicy: ((keyPolicy_KeyExpirationPeriodInDays == 0) ? null : keyPolicy)
largeFileSharesState: largeFileSharesState
minimumTlsVersion: minimumTlsVersion
networkAcls: {
bypass: networkAcls_Bypass
defaultAction: networkAcls_DefaultAction
ipRules: networkAcls_IpRules
resourceAccessRules: networkAcls_ResourceAccessRules
virtualNetworkRules: networkAcls_VirtualNetworkRules
}
publicNetworkAccess: publicNetworkAccess
routingPreference: {
publishInternetEndpoints: routingPreference_PublishInternetEndpoints
publishMicrosoftEndpoints: routingPreference_PublishMicrosoftEndpoints
routingChoice: routingPreference_RoutingChoice
}
sasPolicy: ((sasPolicy_ExpirationAction == '') ? null : sasPolicy)
supportsHttpsTrafficOnly: supportsHttpsTrafficOnly
}
}
output storage_resource_id string = storageAccounts.id
`
When I tried passing '' as suggested It is not considering it as null. Please see screenshot below.

adding comment to track issue.

You can do a hacky workaround to get around this, not ideal but it works:
Keep your param:
@description('Restrict copy to and from Storage Accounts within an AAD tenant or with Private Links to the same VNet.')
@allowed([
  'AAD'
  'PrivateLink'
  ''
])
param allowedCopyScope string = ''

Add a variable with a dictionary:
// Dictionary to use for lookup
var allowedCopyScopeDictionary = {
  AAD: 'AAD'
  PrivateLink: 'PrivateLink'
}

Change your conditional expression to:
allowedCopyScope: ((allowedCopyScope == '') ? null : allowedCopyScopeDictionary[allowedCopyScope])

It seems like the type system needs to understand that a ternary operator is altering the union type that is going into it. Let's say if foo has type 'a' | 'b' | '' and the type of expression foo == '' ? null : foo needs to become null | 'a' | 'b' rather than null | 'a' | 'b' | '' which is used currently.


Flexible allocation hangs when the output of scontrol show is large

$ ./bin/reframe -C config/cscs.py -c ../misc/reframe-tests/flexible/hostname_check.py -r
Command line: ./bin/reframe -C config/cscs.py -c ../misc/reframe-tests/flexible/hostname_check.py -r
Reframe version: 2.15-dev2
Launched by user: karakasv
Launched on host: daint102
Reframe paths
=============
    Check prefix      :
    Check search path : '../misc/reframe-tests/flexible/hostname_check.py'
    Stage dir prefix     : /users/karakasv/Devel/reframe/stage/
    Output dir prefix    : /users/karakasv/Devel/reframe/output/
    Perf. logging prefix : /users/karakasv/Devel/reframe/perflogs
[==========] Running 1 check(s)
[==========] Started on Mon Nov  5 23:16:20 2018

[----------] started processing HostnameCheck (HostnameCheck)
[ RUN      ] HostnameCheck on daint:gpu using PrgEnv-cray

scontrol seems to be hanging for some reason:
$ ps -elf | grep scontrol
0 S karakasv  7892  7880  0  80   0 - 28803 pipe_w 23:16 ?        00:00:00 scontrol -a show -o nodes
0 S karakasv 25636 18079  0  80   0 -  2023 pipe_w 23:29 pts/9    00:00:00 grep --color=always scontrol

$ pstree -u $USER
sshd───bash───python3───scontrol

The problems is caused the use of the wait() and the fact that the output is large enough.

The problem happens due to the large number of lines produced when trying to show all the nodes in a large system such as daint.


Avoid a second traversal of the file table

Instead of re-discovering the files affected by a fast-path edit, store them on the LSPFileUpdates value so that they're available later. The one wrinkle here is that we can't store core::FileRef values as they're not stable between when we determine if the fast-path can be taken, and actually taking it. Instead, we store the original paths of the files, and rediscover their core::FileRef when type-checking on the fast path. This should be significantly faster than re-computing the extra modified files and traversing the file table.
Motivation
Performance on the fast path.
Test plan

This shouldn't change any existing behavior.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_RNrQ7iVzIxCWVY
→ https://go/builds/bui_RNrQYx0nIzFUgV
→ https://go/builds/bui_RNrQF73sy5kUNd


[19.0.x] [WFLY-13022] Upgrade MicroProfile Metrics 2.3

also upgrade smallrye-metrics to 2.4.0
add optional base metrics cpu.processCpuTime

JIRA: https://issues.redhat.com/browse/WFLY-13022
Signed-off-by: Jeff Mesnil<EMAIL_ADDRESS>

Corresponding PR for WildFly 20 is #12989


Where to store config file

Hi,
its a wonderful work here but i was trying to execute the examples given and getting setup error .
Can you please help that where should i be saving my conf.toml file ?
I am saving it in the same folder as of my script. but i am getting exception of setup_Error

There is no restriction as to where you should be saving your TOML config for spdlog_setup, it works the same way as you would do ifstream to open file based on your pwd. You can first try with ifstream to open the file and read out the file string content, and if it works, spdlog_setup::from_file will work with the same given path.
To try to provide additional help, assuming you have your working directory at /path/to/your/dir and your compiled progam is named yourprogram, and your program follows the example to read like this spdlog_setup::from_file("log_conf.toml");, you should have your CMD/bash to be be in /path/to/your/dir first, then execute your program with ./yourprogram (or yourprogram.exe if CMD).
If you are still unable to get it working, you should clarify how where are your executable and config file are located.

No i put it in the same directory (my code and the config file) but i am getting setup error :
#include "spdlog_setup/conf.h"

#include <iostream>
#include <string>

int main() {
    try {
        // spdlog_setup::setup_error thrown if file not found
        spdlog_setup::from_file("log_conf.toml");

        // assumes that root logger has been initialized

        std::cout<<"Printing data\n"<<std::flush;
        auto logger = spdlog::get("root");
        logger->trace("trace message");
        logger->debug("debug message");
        logger->info("info message");
        logger->warn("warn message");
        logger->error("error message");
        logger->critical("critical message");
        logger->info("Print data");
        std::cout<<"Printing data\n"<<std::flush;

        // ...
    } catch (const spdlog_setup::setup_error &) {
        std::cout<<"Printing data in 1st catch\n"<<std::flush; // this statement is printing for me
        // ...
    } catch (const std::exception &) {
        std::cout<<"Printing data in 2nd catch\n"<<std::flush;
        // ...
    }
}

I am compiling as below :
clang++ -o staticConfigFileSetup staticConfigFileSetup.cpp

and my config file resides in the same path as of my code.

Okay then in that case, I will assume that you have your binary + config TOML file + you are running your command with relative path in the same dir as the binary.
I will need you to do these two things before I can help:

Show the content log_conf.toml that you are using that gives the error.
Do the following to show what is the error message in setup_error:

} catch (const spdlog_setup::setup_error &e) {
    std::cout << e.what() << "\n";
} ...

e.what()
Printing data in 1st catch
Sink 'syslog_st' error:
 > Invalid sink type 'syslog_sink_st' found

log_conf.toml

# level is optional for both sinks and loggers
# level for error logging is 'err', not 'error'
# _st => single threaded, _mt => multi threaded
# syslog_sink is automatically thread-safe by default, no need for _mt suffix

# max_size supports suffix
# - T (terabyte)
# - G (gigabyte)
# - M (megabyte)
# - K (kilobyte)
# - or simply no suffix (byte)

# check out https: // github.com/gabime/spdlog/wiki/3.-Custom-formatting
global_pattern = "[%Y-%m-%dT%T%z] [%L] <%n>: %v"

[[sink]]
name = "console_st"
type = "stdout_sink_st"

[[sink]]
name = "console_mt"
type = "stdout_sink_mt"

[[sink]]
name = "color_console_st"
type = "color_stdout_sink_st"

[[sink]]
name = "color_console_mt"
type = "color_stdout_sink_mt"

[[sink]]
name = "file_out"
type = "basic_file_sink_st"
filename = "log/spdlog_setup.log"
# truncate field is optional
# truncate = false (default)
level = "info"
# optional flag to indicate the set - up to create the log dir first
create_parent_dir = true

[[sink]]
name = "file_err"
type = "basic_file_sink_mt"
filename = "log/spdlog_setup_err.log"
truncate = true
level = "err"
# to show that create_parent_dir is indeed optional(defaults to false)

[[sink]]
name = "rotate_out"
type = "rotating_file_sink_st"
base_filename = "log/rotate_spdlog_setup.log"
max_size = "1M"
max_files = 10
level = "info"

[[sink]]
name = "rotate_err"
type = "rotating_file_sink_mt"
base_filename = "log/rotate_spdlog_setup_err.log"
max_size = "1M"
max_files = 10
level = "err"

[[sink]]
name = "daily_out"
type = "daily_file_sink_st"
base_filename = "log/daily_spdlog_setup.log"
rotation_hour = 17
rotation_minute = 30
level = "info"

[[sink]]
name = "daily_err"
type = "daily_file_sink_mt"
base_filename = "log/daily_spdlog_setup_err.log"
rotation_hour = 17
rotation_minute = 30
level = "err"

[[sink]]
name = "null_sink_st"
type = "null_sink_st"

[[sink]]
name = "null_sink_mt"
type = "null_sink_mt"

# only works for Linux
[[sink]]
name = "syslog_st"
type = "syslog_sink_st"
# generally no need to fill up the optional fields below
# ident = "" (default)
# syslog_option = 0 (default)
# syslog_facility = LOG_USER (default macro value)

# only works for Linux
[[sink]]
name = "syslog_mt"
type = "syslog_sink_mt"
# generally no need to fill up the optional fields below
# ident = "" (default)
# syslog_option = 0 (default)
# syslog_facility = LOG_USER (default macro value)

[[pattern]]
name = "succient"
value = "%c-%L: %v"

[[logger]]
name = "root"
sinks = [
    "color_console_mt",
    "rotate_out", "rotate_err"]
level = "trace"

[[logger]]
name = "console"
sinks = ["console_st", "console_mt"]
pattern = "succient"

I will be assuming that you are on Linux, since syslog is not supported on Windows (OSX should be fine I think?)
Based on the error message: "Invalid sink type 'syslog_sink_st' found", the most likely reason is that your spdlog_setup is not from the latest tag v0.3.0-alpha.2 (or latest commit, whichever is fine).
There was a breaking change between v0.3.0-alpha.1 and v0.3.0-alpha.2; v0.3.0-alpha.1 only supports pre-v1.X spdlog and uses syslog_sink instead of syslog_sink_st (see: https://github.com/guangie88/spdlog_setup/tree/v0.3.0-alpha.1#static-file-configuration, and search for syslog_sink), while v0.3.0-alpha.2 uses both syslog_sink_st and syslog_sink_mt (which follows the same example as the current master README.md). So in order to fix the problem, do make sure you are using either the latest commit of spdlog_setup, or use v0.3.0-alpha.2.
In any case, you might not need syslog sink (or the many other types of sink in the example) to begin with, so it will also work if you just remove syslog sinks from your TOML config.


Nuget package

Could you please create and upload to nuget.org an updated nuget package based on this 2nd latest commit https://github.com/tekmaven/NServiceBus.Persistence.MongoDb/commit/59b305b79f3a7782ca9412bac8dcf7d2f2004f3c to get a fix for issue #8 (using nsb 5.2.14) Thank you!

This is released as a part of NServiceBus.Persistence.MongoDb 8.0.24.


[moleculer-bull] Multiple named worker on same queue

Fix #34

Pull Request Test Coverage Report for Build 301

12 of 12 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.08%) to 96.633%

Totals

Change from base Build 299:
0.08%

Covered Lines:
219

Relevant Lines:
219

💛  - Coveralls

Published


Faster/More efficient duplicate removal for exact/fuzzy dedup.

Is your feature request related to a problem? Please describe.
The current deduplication examples suggest compute on the list of duplicate documents produced via exact/fuzzy deduplication and use the computed list to filter out input documents. This doesn't work in cases where the duplicate list is too large and doesn't fit on the client.
Ideally curator can provide additional classes/methods to remove duplicates from the list of duplicates list more efficiently.
Describe the solution you'd like
A broadcast merge approach like the one suggested by @VibhuJawa works good enough at the 4-8TB scales where the duplicate list is small enough to be broadcasted to each worker and is worth implementing first.
Longer term there might be a need for smarter partitioning of the duplicate list so that different files/subset can handle their own list of duplicates differently.
Describe alternatives you've considered
N/A
Additional context
The Zyda-2 tutorial and pre-training data tutorial both contain alternate approaches to compute since it's memory intensive.

A broadcast merge approach like the one suggested by @VibhuJawa works good enough at the 4-8TB scales where the duplicate list is small enough to be broadcasted to each worker and is worth implementing first.

Examples of removing using merge:
https://gist.github.com/VibhuJawa/7c780209bdcad9ac7615bd84b86cde58

Longer term there might be a need for smarter partitioning of the duplicate list so that different files/subset can handle their own list of duplicates differently.

My best suggestion here if we want to skip doing a broadcast merge is to do a batched index merge (like we do in the  CC stage) for this, i think thats most scalable.
We can also do this based on a heuristic as the list of IDs is all ready in distributed GPU memory, so we can switch b/w the two, this will mean we dont compromise on perf on the short term
We can also play tricks on filtering if we wanted to by first creating a map of file->dataset_ids and then doing a filter on a dataset.

https://gist.github.com/VibhuJawa/749045921b9e5a81b42a4b41cd4b03dc


fix parts of the bsky theme

🔧 What does this fix? 🔧
Allows the bluesky theme to apply again after changes to the currently used theme selector
also fixes latte (i hope) by adding selectors for latte-specific things
🗒 Checklist 🗒

[x] I have read and followed Catppuccin's contributing guidelines.
[x] I have updated the version appropriately in the ==UserStyle== header of the catppuccin.user.css file.

grr i will eat you


[PE-D] Remarks cannot be longer than window width

Cannot see much of the remarks as anything longer than my window width, the remark is cut off and there is no function for me to see the full remarks. As a user, since i have to keep track of so many friends, i certainly cannot remember what i wrote.
Screen Recording 2021-10-29 at 16.51.46.mov

Labels: type.FeatureFlaw severity.Medium
original: sheehui/ped#4

This issue should be fixed as long as #149 is merged.


Bump make test timeout due to CI reaching it

[ ] I added CHANGELOG entry for this change.
[x] Change is not relevant to the end user.

Changes
Bump go test timeout in the make test target from the default (10m) to 15m, because I noticed some CircleCI test runs were timing out: https://app.circleci.com/pipelines/github/thanos-io/thanos/12838/workflows/94c287d3-8c0e-47fc-ab33-f4718c4fde8b/jobs/22657.
Verification

dang! we could well served to either further parallelize tests or simplify the test suite so that CI runs in under 15 minutes

100% agreed, @squat. I think the easier path will be to parallelize into more machines.  Hopefully the test runtime reduction will follow the amount of test runners. 😄


Error when run production build use custom server with express.

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior
I try to do

Run production build use custom server with express
Jest Test

Current Behavior

I can run next in development mode with node server.js
Jest Test ( Jest is ok, run normally)

After next build, I try to start custom server with express with
NODE_ENV=production node server.js
This is log
TypeError: Cannot read property 'toLowerCase' of undefined
    at a.renderDOM (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/react-dom/cjs/react-dom-server.node.production.min.js:36:391)
    at a.render (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/react-dom/cjs/react-dom-server.node.production.min.js:36:228)
    at a.read (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/react-dom/cjs/react-dom-server.node.production.min.js:35:250)
    at renderToStaticMarkup (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/react-dom/cjs/react-dom-server.node.production.min.js:44:75)
    at _callee3$ (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/next/dist/server/render.js:223:100)
    at tryCatch (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/regenerator-runtime/runtime.js:62:40)
    at Generator.invoke [as _invoke] (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/regenerator-runtime/runtime.js:296:22)
    at Generator.prototype.(anonymous function) [as next] (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/regenerator-runtime/runtime.js:114:21)
    at step (/home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /home/naijab/freelance/neogamer/frontend/gamerui/src/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
    at <anonymous>

Steps to Reproduce (for bugs)

yarn build --> next build
yarn start --> NODE_ENV=production node server.js
Point a browser to http://localhost:3000
Show 500 errors flash An unexpected error has occurred.

Context
When start production build, internal server errors cause an unwanted flash of content.

Your Environment

Tech
Version

next
5.0

node
8.9

OS
Ubuntu

browser
Chrome 64

etc

This error is usually because you tried to render a component the is actually undefined. this can be because you tried to import a component that doesn't exists. Please check that out, it's impossible to reproduce without seeing your code.

I am having this issue too now, just can't see what is causing this, AFAIK I have no missing components.


Include onchain gas cost

Closes #1186
Introduced changes

onchain data gas is computed from the state change from blockifier state

Checklist

[x] Linked relevant issue
[x] Updated relevant documentation
[x] Added relevant tests
[x] Performed self-review of the code
[x] Added changes to CHANGELOG.md

I  think we should add some explicit test cases for that, and see if it complies with the formula that was provided in the docs

Tests are added. Everything complies with the docs


Please evaluate codeBooga

CodeBooga-34B-v0.1
https://huggingface.co/oobabooga/CodeBooga-34B-v0.1
Thanks!

@sabagithub Please find results here:

Overall it seems to stay between the two.
Also cc the original author @oobabooga if the results are informative. Let me know if anyone wants me to put this on the leaderboard as well.
Also FYI, now ppl can request an independent model evaluation through: https://github.com/evalplus/evalplus/issues/new/choose

Thanks for the eval @ganler, that's good to know.

Wow, that was quick @ganler! Thank you for processing it, and for the new link
It's strange to me because although PhindV2 gives compiling code as often as CodeBooga, CodeBooga seems to follow the specs a little better. Maybe it has better reasoning skills... Anyway, thanks again!

Wow, that was quick @ganler! Thank you for processing it, and for the new link
It's strange to me because although PhindV2 gives compiling code as often as CodeBooga, CodeBooga seems to follow the specs a little better. Maybe it has better reasoning skills... Anyway, thanks again!

I saw a few cases where the merged model did not import required libraries inline but instead did that after the whole function. Anyways HumanEval+ is just one of the many aspects to evaluate LLM coders. The new model may show better results on other benchmarks or human evaluation. 😃


First UI tests (fixes #289)

Signed-off-by: Josef Kopriva<EMAIL_ADDRESS>

@jrichter1 please review and share your thoughts :)

LGTM


"Downloader" in header1.png is misspelled

In the second preview image called header1.png in README.md, word Downloader is misspelled as Downoader. Direct image link.
I suppose the image should be redone in whatever software it originally was done in and re-uploaded.

Thanks a lot


added nav with two sections and a footer

Looks good. Thank you!

Looks good. Thank You.
closes #1


Help - IronPython

From @wta12 on May 22, 2018 8:26

VSCode Version: 1.23.1
OS Version: Window 10 Education x64

Steps to Reproduce:
"type": "python",
"request": "launch",
"name": "Python35",
"program": "${file}",
"cwd": "${fileDirname}",
"showDebuggerOutput": true

i run debug with shown configuration and get this cmd:
G:\PyRevitRun>cd g:\PyRevitRun && cmd /C "set "PYTHONIOENCODING=UTF-8" && set "PYTHONUNBUFFERED=1" && python "C:\Users\User.vscode\extensions\ms-python.python-2018.4.0\pythonFiles\PythonTools\visualstudio_py_launcher.py" g:\UserWork_Folder\Revit_2017\PyRevit\PyRevitRun 54329 34806ad9-833a-4524-8cd6-18ca4aa74f14 RedirectOutput,RedirectOutput g:\Work_Folder\Revit_2017\PyRevit\PyRevitRun\pyWinUI.py "
VS code auto moves to CMD Terminal Tab - i want to keep at debug console for better viewing - is there any setting for auto cleaning at CMD Terminal Tab every time i run debug cmd

Does this issue occur when all extensions are disabled?: No
Thanks
Copied from original issue: Microsoft/vscode#50267

Sorry, there's no setting to automatically clear out the terminal when you start debugging.

So how to keep viewing debug console (always clear out) instead of terminal with VS Code ?

@wta12 I don't think there is a way.


Use microphysics sources from previous timestep

Right now in diagnostic EDMF we are computing and applying the source terms inside the integral. This is causing numerical issues. We can fix them temporarily by setting the precipitation relaxation timescale to 10dt, instead of the default dt.
Alternatively we could try computing the source terms after the big integral is done. This would mean we would be applying the previous timestep sources inside the integral.
It's worth testing which of the two bad options performs better

Here it is how it looks for box TRMM. Based on those results I think we should keep what we have right now. Applying the previous time step source term is even more of a hack and it does not help us to set the precipitation relaxation timescale back to dt. cc: @szy21 @tapios

Applying the current source term inside the diagnostic EDMF integral with precipitation timescale = 10xdt (main branch)

Applying the current source term inside the diagnostic EDMF integral with precipitation timescale = dt

Applying the previous timestep source term inside the diagnostic EDMF integral with precipitation timescale = 10xdt

Applying the previous timestep source term inside the diagnostic EDMF integral with precipitation timescale = dt

Thanks for looking at this. Let's keep the longer relaxation timescale then.

Thanks for looking at this. Let's keep the longer relaxation timescale then.

Agreed. I'm glad that the previous timestep sources hack is not better


BREAKING CHANGE: Use requestImageForAsset to retrieve UIImage and remove includeExtra (as it will be default)

[X] Explain the motivation for making this change.
[X] Provide a test plan demonstrating that the code is solid.
[X] Match the code formatting of the rest of the codebase.
[X] Target the master branch, NOT a "stable" branch.

Motivation (required)
This is a fix attempting to remedy issues with file copying silently failing on iOS. We have experienced over 1000+ instances reported through Sentry of files being missing after being picked. This seems to be related to some combination of Live Photos, photos which adjustments, or photos from iCloud. The issue is related to this code:
// Matches both com.apple.live-photo-bundle and com.apple.private.live-photo-bundle
if ([identifier containsString:@"live-photo-bundle"]) {
    // Handle live photos
    identifier = @"public.jpeg";
}

The updated code will handle Live Photos correctly via the requestImageForAsset method which handles photos, Live Photos and videos (as thumbnails).
Test Plan (required)
The test plan is to use this branch in our production application for our next release. Using Sentry to determine if this reduces the frequency of the issue.
Notes
This removes includeExtra and treats PHAsset as a first-class citizen, aka. it's a breaking change as to access the library on iOS 14 you would need to have library permissions. This is to simplify the API, but also a preparation for moving over more metadata to rely on PHAsset in subsequent PRs.
Fixes #1541 (for images only, not videos), #1876, #1864

Nice job, @lukebrandonfarrell! Looking forward for this to be reviewed & patched! 😄

We've been using this in production for about a year now, I'd say it's good to merge, let me rebase the files

@Johan-dutoit As mentioned above, we've been using this in production for almost a year, and we've seen a drop-off in these issues. Could I get a quick review then can merge this?

Yes please!

👋  Some more conflicts.  Hoping to test and merge it this weekend or next.

@Johan-dutoit any update on this?

Happy to merge once conflicts resolved.


CentOS-7 End of Life

After June 30, 2024 which is the EOL of CentOS-7, what are we supposed to do for the installation of AWIPS? Should we keep using CentOS-7, or can we migrate to other OS?

We will be using Rocky8 for our next release. We will have a CAVE release out tomorrow to connect to our public EDEX.
However, the National Weather Service is still working on their 23.4.1 release to operations. In the interim, we are waiting on final clearance from the NWS before releasing a public Unidata EDEX install and source code. We understand the CentOS7/RHEL7 end of life is upon us, but this is all we are allowed to provide at this time.

May I know if the type/version of Linux matters in case of using Docker for installation?

If running in Docker, the base OS should not matter.

@tiffanycmeyer13 are you able to release the v23 source for the non-EDEX parts of AWIPS?

@tiffanycmeyer13 are you able to release the v23 source for the non-EDEX parts of AWIPS?

At this point we are not able release any source code for this version of AWIPS. Once we get the OK from NWS we will.

May I know if the type/version of Linux matters in case of using Docker for installation?

CAVE is designed for RHEL8 flavors of Linux (including Rocky)


fix(STONEINTG-248): don't specify environment for ITS

Description

Remove the environment configuration from ITS spec
Full environment testing will be added in future (STONEINTG-398)

Issue ticket number and link
STONEINTG-248
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist:

[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added meaningful description with JIRA/GitHub issue key(if applicable), for example HASSuiteDescribe("STONE-123456789 devfile source")
[ ] I have updated labels (if needed)

/retest-required

/retest-required

/test redhat-appstudio-e2e

cluster related issue, not related with ur changes


Tree/Recursive struct support in thrift

Patches to implement tree/list/co-recursive structures in thrift, a la protobufs.

Anyone else facing that issue: workaround is to declare a reference:
struct RecSelf {
   1: i16 item
   2: optional RecSelf &self
 }

Note the &.


Add test.toolchain field

Enables running cargo compete test with old Rust without Rustup Override.
Since rust-analyzer no longer works under Rust 1.42 environment, we have to explicity specify +1.42.0 for testing.

@lnicola Thank you for pointing it out. I will think about it.


Support for the list organization repositories endpoint

Codecov Report

Merging #162 into master will increase coverage by 0.12%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #162      +/-   ##
==========================================
+ Coverage    87.5%   87.62%   +0.12%
==========================================
  Files          40       40
  Lines         600      606       +6
  Branches        4        3       -1
==========================================
+ Hits          525      531       +6
  Misses         75       75

Impacted Files
Coverage Δ

...4s/shared/src/main/scala/github4s/GithubAPIs.scala
100% <100%> (ø)
:arrow_up:

...in/scala/github4s/free/algebra/RepositoryOps.scala
69.23% <100%> (+2.56%)
:arrow_up:

...b4s/shared/src/main/scala/github4s/api/Repos.scala
96.77% <100%> (+0.34%)
:arrow_up:

...cala/github4s/free/interpreters/Interpreters.scala
77.46% <100%> (+0.32%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fdea87a...660ac75. Read the comment docs.

@juanpedromoreno woops thanks for calling it out, I had completely forgotten about the doc in this one.
Will add it to his PR :+1:


Clear debug process identifiers array at stop() to prevent invalid checking of them

Processes are killed but an array of theirs identifiers is not cleared after that. Should fix https://github.com/godotengine/godot/issues/47087

Thanks!

Should this be cherry-picked to 3.x and 3.4? See https://github.com/godotengine/godot/issues/42251#issuecomment-1088915611.

I'm not sure this is applicable to 3.x, but I can check.

This doesn't apply to 3.x, it only has a single pid.


Make all key properties have the GenerateValueOnAdd flag set by convention

We are making a change so that the default behavior for properties that are part if keys is to have value generation enabled. Several points:

Even with this flag set values will only be generated if the value of the property has not been set (is the default) when the entity is added. This means that use of non-generated keys where values have been set should not be affected.
There is chance of collision of real key space and temporary key space, but we think this is unlikely to be a problem in the wild. Where collisions might occur they can be avoided by always setting key values explicitly so value generation never kicks in or by configuring different value generators that do not collide.
Always generating keys provides a more flexible experience for dealing with disconnected graphs since temporary values will be used which can then be changed before saving
An exception will be thrown if key generation is needed but no key generator is configured This can be avoided by setting the values explicitly before add or adding a new value generator.
An exception will be thrown if a temporary key value is still set when the state of the object is changed to Unchanged, Modified, or Deleted. This will happen if SaveChanges is given temp values and doesn't generate real values.

A couple of notes:

This needs to be done for any property that is a principal key, not only for primary keys.
We could consider changeing Property.DefaultGenerateValueOnAdd to:

protected virtual bool DefaultGenerateValueOnAdd => this.IsKey();

Or even to something more complicated that determines if this is the root principal in the chain. This is similar to what we do for IsReadOnly. However, it could be making the core metadata too magic/convention-based and hence we may not want to do it this way, and we may want to consider not doing IsReadOnly this way as well.

Another note. Currently is a property has GenerateValueOnAdd set and is not marked as using a SQL Server sequence, then it will be assumed to be SQL Server Identity. This doesn't work because properties such as alternate principal keys need to have GenerateValueOnAdd set but should not be treated as SQL Server Identity by either Migrations or the update pipeline.
It seems like there should be a SQL Server-specific convention which determines which column, if any, should be an Identity column and annotate appropriately. Migrations should do nothing other than honor the annotation. The value generator selector should use a temporary generator for any property that is not marked as using a sequence. The update pipeline should do nothing for columns that are not annotated as Identity. If temp values still exist in columns other than the Identity column then the existing exception for this case is correct.

fixed via 5201798fe95d94e4c66f79f58ad4ad62a3ca878a


[GitHub] Bug: Update [p]github whatlinks Instructions

Not a bug per say, but something I noticed while playing around with it and it's not a pressing matter.
In the github whatlinks description for a private repository, it says to search for the atom link in the element inspection for the browser. It took a bit of digging around, but I think it would be good to note that in order to see this atom link, you'd need to view the Commit History for a particular branch.
When I saw in #2 about the different branches, I believe simply just going to the commit history for a selected branch would suffice to get those specific commits. Looks like the basic setup is this:
https://github.com/<Username>/<repository>/commits/<branch>.atom?token=<token>

This should help a bit to try and make things a bit clearer, possibly even to the end-user who wants to use the cog to set it up on their own. As it stands currently, going to the repo and searching in the element inspection for something in regards to atom doesn't return any results, which lead me to search around to find something in the actual commit history.

@TwinDragon Thanks for bringing this to my attention, when I first created this cog a few months ago (it was private for quite a bit before I started my repo), those instructions worked and the .atom file could be found in the root repo link. I just checked myself too and it has indeed moved; however, I am hesitant about correcting it immediately as I haven't looked at this cog for a while, and am unsure of how it would handle different branches, etc. which this may affect if users go to a different branch's .atom. That's one of the updates I am working on, if I get more time before then to look then I will correct this beforehand but am too busy right now (hopefully in the next week or two).

Yeah, not a problem. It's nothing pressing, just an observation whole I was digging around.

This has now been fixed with basically a rewrite of the cog in 4e1da009626abca115aba9163df5730eae485ad3.


Attached file won't open in vscode

Vinduer.zip

VSCode Version: Code 1.6.1 (9e4e44c19e393803e2b05fe2323cf4ed7e36880e, 2016-10-13T16:21:53.542Z)
OS Version: Windows_NT ia32 6.1.7601
Extensions: donjayamanne.python-0.3.24, robertohuertasm.vscode-icons-3.0.0, Tyriar.sort-lines-1.1.0, wmaurer.change-case-0.4.1

Steps to Reproduce:

Open the attached file in vscode
Message: "The file will not be displayed in the editor...". It works fine in Sublime and TextPad.

@ulvtro the file seems to be encoded with UTF-16 but not using a BOM. We can only show UTF-16 if the file is saved with a BOM. See http://unicode.org/faq/utf_bom.html


Awkwardness in the IconData API

Describe the bug

To Reproduce
Steps to reproduce the behavior:

Expected behavior

Screenshots

Desktop (please complete the following information):

OS:
Browser
Version

Smartphone (please complete the following information):

Device:
OS:
Browser
Version

Additional context

Left this open in a background tab and it somehow posted itself?


fix: radiofrance, #3077

I can confirm the fix is working, but unfortunately it only returns the first page of results (so 12 items max). This is still a bit better than the official feed (9 items max) but still, it's not the whole list :confused:


On monitor disconnect, don't change which workspace is on monitors that didn't get disconnected

Steps to reproduce:

Connect 2 monitors
Put some content on each of them
Disable one of them with hyprctl
The monitor that didn't get disabled now has a different workspace on it!

It would be great if that stayed consistent. I'm happy to upload any debug info or a video showing the bug if that's helpful.

I second that improvement

having the same problem, currently i have 6 persistent workspaces,
1,2 on ist monitor,  3-6 on 2nd monitor
when wake happens from suspend, ist monitor automatically goes to 8th workspace

the same problem, currently i have 6 persistent workspaces,
1,2 on ist monitor, 3-6 on

this got solved on declaring a workspace default for a display


.*: fix bug introduced by https://github.com/pingcap/tiflash/pull/7175

What problem does this PR solve?
Issue Number: close #7229
Problem Summary:
What is changed and how it works?
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-unit-test

/run-unit-test

/run-unit-test

/merge