[WIP] basic move resize

A move-resize operation must be handled asynchronously because it is done with a request to the shell surface and we need to wait for it to update its buffer before we can update the view size. Sometimes, updating the size of the view must be done at the same time as updating one of the view x/y coordinates so that must be kept in sync.
That's what the view move_resize function does. It takes an Area of the new position of the view, makes a set_size request on the toplevel, and then does both the x/y position change and the width/height update at the same time. Without that, you would update x/y first and width/height at some later time causing a flash of partially rendered content.
The view now has a PendingMoveResize struct which is set in the view.move_resize function and read in the surface commit handler. On commit, if the configure serial matches the PendingMoveResize serial, it does the update.
The seat got a new mode called Resizing. Here we store the offset of the cursor on the surface, and the original position of the view. We calculate how much the cursor has moved from the original spot and adjust the final view position based on that.
TODO:

[x] async move/resize
[x] seat resize state
[ ] bug: jitter in the bottom corner on draging the top-left corner and moving rapidly (exists in rootston as well) that sort of looks like a rounding issue.
[x] put the WLR_EDGE enum somewhere (maybe in wlroots-rs).
[ ] can this be simpler? (maybe not).

I dunno if I can fix that jitter. It doesn't seem to be a big deal anyway. Maybe leave that task up to someone who it bothers in the future.

Thanks!


Unable  to upload video file.

I'm trying to upload video file using content, but not been able to upload. But the server is showing the file size of only 15kb only (Looks like the metadata of file but not sure).
I'm using QBContent.uploadFileTask API to upload it.

Upload video file is done. Will be available in upcoming release build.


Turborepo support

closes #456
This PR adds Turborepo support to Nixpacks.

[x] Tests are added/updated if needed
[x] Docs are updated if needed

Could you fix conflicts please?

new version 1.5.1 -> https://turborepo.org/docs/handbook/deploying-with-docker#the-solution prune support

@aleksrutins any updates?

Any docs for this?

Any docs for this?

See: https://nixpacks.com/docs/providers/node

Has it been tested with PNPM? I'm getting a weird error during the build phase


WIP Calculate minimum ADA value

This code is from cardano-wallet and needs a lot of cleanup.

Example:
$ cardano-cli transaction calculate-min-value --multi-asset "5<PHONE_NUMBER>0000000000000000000000000000000000000000000000.couttscoin + 6<PHONE_NUMBER>0000000000000000000000000000000000000000000001.jimbocoin" --protocol-params-file params.json
Result Lovelace 1703702

Fixes https://github.com/input-output-hk/cardano-node/issues/2571

bors merge

bors merge

This adds a new command that calculates the "minimum ADA value" (sometimes known as the minimum deposit): https://developers.cardano.org/en/development-environments/native-tokens/multi-asset-tokens-explainer/#:~:text=Ada is Cardano's principal currency,currently set at 1 ada).


setTimeout Warning

Hi, I just cloned, installed, started and executed this project, and I received the setTimeout warning on my Android 7.1.
I searched in the whole project for the setTimeout and is only used in the logout. I commented this line and the warning continuous showing up, so the problem is not there.
The warning say: Setting a timer for a long period of time, i.e. multiple minutes, is a performace and correctness issue on Android as it keeps the timer module awake, and timers can only be called when the app is in the foreground. (Saw setTimeout with duration 143614ms)

@dv297 is correct. This is a long standing issue from Firebase.
https://github.com/mcnamee/react-native-starter-kit/issues/84


client: Add forgotten getByID and getByScriptID

@zziger
I built and test this PR and works without any problem.
Merge it, please.


HAProxy PROXY protocol v1/v2 support

What is the problem your feature solves, or the need it fulfills?
its a quite common usecase to have pingora behind a loadbalancer when running inside of cloud infrastructure. They usually implement the PROXY protocol to give information about the TCP session like the connection initiator, connection target, client & server and the proxy. It would be immensly helpful to have those informations on pingora's side.
Describe the solution you'd like
Implement the PROXY protocol as decribed in the docs from HAProxy itself. Otherwise there are some rust based implementations floating around that could speedup the process.
example 1
example 2
example 3
example 4
Describe alternatives you've considered
The other solution would be to use mmproxy but that makes it quite inflexible and brings other challenges in itself.
Thank you for this awesome project. 👍

We need this for on premise infrastructure . Currently using HAPROXY with proxy protocol in front of k8s ingresses


Added the account name by field

Type of change
Note: Fill x in []

[ ] bug
[x] enhancement
[ ] documentation
[ ] dependencies

Description
Modify the GCP billing according to the billing projects.
Before:

After:

For security reasons, all pull requests need to be approved first before running any automated CI

I dont see any difference, what do exactly add ?


Roll/Mint should just go to connected address without manual user input

The original Mint component takes Recip Address and ipfs hash.

removed the IPFS field
but left original way to mint below.

"rollToMint" works as expected (without requiring input).


couple RA tests fail if built with cranelift

´rustc 1.79.0-nightly (83d0a940c 2024-04-06)´
I noticed that when I run a cargo test of rust-analyzer @ d9c29afaee6cb26044b5a605e0073fcabb2e9722
, all the tests pass, but when I use cranelift, some some of the tests fail:
RUSTFLAGS="-Zcodegen-backend=cranelift" ~/.cargo/bin/cargo +master test
failures:
    consteval::tests::const_loop
    consteval::tests::try_block
    consteval::tests::try_operator
    layout::tests::recursive
    tests::regression::unsize_array_with_inference_variable
    tests::simple::async_fn_and_try_operator
    tests::simple::block_modifiers_smoke_test
    tests::simple::recursive_type_alias
    tests::traits::infer_try_block
    tests::traits::infer_try_trait
    tests::traits::trait_impl_self_ty_cycle
    tests::traits::unselected_projection_in_trait_env_cycle_2

test result: FAILED. 753 passed; 12 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.25s

consteval::tests::const_loop is because of a std::panic::resume_unwind thrown on cycle errors, which because cg_clif only supports panic=abort currently can't be caught again. The other tests likely have the same issue.


docset appears to be somewhat stale

https://github.com/Kapeli/Dash-User-Contributions/tree/master/docsets/RFCs shows no commits for the last eleven months, and indeed I see no RFCs after 7240 in it - IETF is at 7538 now.
Can you update the docset? If not, please let me know and I'll give it a shot.

local repo has been updated and v1.2 tagged.  Pull request sent to Kapeli/Dash-User-Contributions

Thanks, much appreciated!

update as been merged in Dash-User-Contributions.

Thanks again :+1:


Is this still use with Faceapi.js?

you can, but i suggest https://github.com/vladmandic/human, its much newer and has this built-in.


Dropdown options display should reset with onBlur event

added new feature along with fixing this issue: when typing, the dropdown can automatically hover on the first option that matches the input (e.g., for states dropdown, typing al will hover on Alabama). So when users hit enter the option on hover will be selected.

new bug: when entering random stuff that does not match any option and hit enter, code breaks
problem: did not check if this.state.options is an empty array before calling it in const newValue = this.state.options[this.state.tempIndex].props.
fix: add new check to make sure this.state.options is not an empty array


Types support

Fix #25

@SeriyBg the only comment from my side is https://github.com/hazelcast/hazelcast-jdbc/pull/29#discussion_r566642499

@SeriyBg the only comment from my side is https://github.com/hazelcast/hazelcast-jdbc/pull/29#discussion_r566642499


Allow wand to pull from backpacks and other inventories

Building Gadgets can pull blocks from inventories carried by the player such as backpacks, dank storage, and black hole talismans. This is convenient as it cuts down on inventory management.
It may be possible to copy the code from Building Gadgets since it's under the MIT licence.

Construction Wands can do that, too. I have explicitly tested the Botania black hole talisman and dank storage units and they both work.
In fact, all mod items that implement the Forge ItemHandler capability work with CW.
Botania has its own API which is supported as well.

If there is a specific item container that is not supported, please mention it (plus mod + minecraft version) and I can look into it.

@Theta-Dev Does it make sense that the wand can pull from a Sophisticated Backpacks backpack that's in my inventory, but can't pull from it when I'm wearing it in a Curios slot?
That's not intentional, I hope?
ConstructionWand: 1.18.2-2.7
Sophisticated Backpacks: <IP_ADDRESS>8
(in ATM7-TTS modpack)


osquery_status logs are missing timestamps

The log lines in the osquery_status logfile don't contain timestamps and are unlikely able to be indexed by many centralized logging platforms
What version of fleet are you using (fleet version --full)?
1.0.5

Can you try running an osqueryd with --tls_dump and see if it is sending logs with timestamps? I'd like to understand whether we are stripping them or if they are not being provided at all. Thanks!

seeing this from tls_dump:
{
  "node_key": "PKOxagYeAtFl3pfYzYwK+GPjICmYiFmh",
  "log_type": "status",
  "data": [
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:53 2017 UTC",
      "unixTime": "1513027913",
      "severity": "0",
      "filename": "init.cpp",
      "line": "379",
      "message": "osquery worker initialized [watcher=9862]",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    },
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:53 2017 UTC",
      "unixTime": "1513027913",
      "severity": "0",
      "filename": "rocksdb.cpp",
      "line": "132",
      "message": "Opening RocksDB handle: /var/folders/02/k62z5c796fs5gm651jc610v80000gn/T/tmp.mgylcnUm/osquery.db",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    },
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:54 2017 UTC",
      "unixTime": "1513027914",
      "severity": "0",
      "filename": "interface.cpp",
      "line": "338",
      "message": "Extension manager service starting: /var/osquery/osquery.em",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    },
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:54 2017 UTC",
      "unixTime": "1513027914",
      "severity": "0",
      "filename": "tls_enroll.cpp",
      "line": "63",
      "message": "TLSEnrollPlugin requesting a node enroll key from: https://localhost:8080/api/v1/osquery/enroll",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    },
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:54 2017 UTC",
      "unixTime": "1513027914",
      "severity": "0",
      "filename": "system.cpp",
      "line": "256",
      "message": "Using host identifier: FA01680E-98CA-5557-8F59-7716ECFEE964",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    },
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:54 2017 UTC",
      "unixTime": "1513027914",
      "severity": "0",
      "filename": "tls.cpp",
      "line": "208",
      "message": "TLS/HTTPS POST request to URI: https://localhost:8080/api/v1/osquery/enroll",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    },
    {
      "hostIdentifier": "FA01680E-98CA-5557-8F59-7716ECFEE964",
      "calendarTime": "Mon Dec 11 21:31:54 2017 UTC",
      "unixTime": "1513027914",
      "severity": "0",
      "filename": "tls.cpp",
      "line": "208",
      "message": "TLS/HTTPS POST request to URI: https://localhost:8080/api/v1/osquery/config",
      "version": "2.10.3",
      "decorations": {
        "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
        "hostname": "kl.groob.io"
      }
    }
  ]
}

and /tmp/osquery_status
{
  "severity": "0",
  "filename": "tls.cpp",
  "line": "208",
  "message": "TLS/HTTPS POST request to URI: https://localhost:8080/api/v1/osquery/enroll",
  "version": "2.10.3",
  "decorations": {
    "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
    "hostname": "kl.groob.io"
  }
}
{
  "severity": "0",
  "filename": "tls.cpp",
  "line": "208",
  "message": "TLS/HTTPS POST request to URI: https://localhost:8080/api/v1/osquery/config",
  "version": "2.10.3",
  "decorations": {
    "host_uuid": "FA01680E-98CA-5557-8F59-7716ECFEE964",
    "hostname": "kl.groob.io"
  }
}

Thanks for the quick triage, @groob!

Clearly looks like we're stripping useful info. Thanks for the repro.

I believe this isn't actually a bug, because until recently there was no timestamp sent by osquery. We recommended using decorators for this in the past.
it looks like osquery decorates status with timestamp by default now, but we dont' track it
https://github.com/kolide/fleet/blob/45165aa29aae52346ed1c458f19b89947d315c61/server/kolide/osquery.go#L76-L83
We should update the status writer to be the same as we did with results in https://github.com/kolide/fleet/pull/1636

Agree with the proposed strategy re https://github.com/kolide/fleet/pull/1636.


support for Cambricon MLUs device

Currently,Transformers and Accelerate have supported cambricon mlu (https://github.com/huggingface/transformers/pull/29627, https://github.com/huggingface/accelerate/pull/2552).
This PR enables users to leverage the cambricon mlu for training and inference with safetensors.

@Narsil Hi, could you help to review this PR?


⚠ Bofrost.es has degraded performance

In 4c7a116, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1306 ms

Resolved: Bofrost.es performance has improved in ee7a636 after 28 minutes.


Handling of special characters

Characters disallowed by the S3 rules for object keys should be removed from the object (renaming it) before updating the list.

Not implementing at this time...would add additional complexity in the bucket setup in order to avoid recursive invocations.


[Android] onDeepLink result data is a string instead of an object.

Report
Plugin Version
6.1.41
On what Platform are you having the issue?
Android
What did you do?
Add callback to onDeepLink.
What did you expect to happen?
I expect the callback to be called with a result object that has a data property that is also an object.
(just like on iOS, and also to be consistent with other callbacks that have a result with data.
What happened instead?
The type of data is string instead.
Please provide any other relevant information.
res => { console.log(typeof res.data) } // object on iOS, string on Android

Perhaps the solution is to just remove toString() here? https://github.com/AppsFlyerSDK/appsflyer-react-native-plugin/blob/afd390eeba54fbbb5fd3780827495d5be4601a8c/android/src/main/java/com/appsflyer/reactnative/RNAppsFlyerModule.java#L180

@curiousdustin thank you. will be fixed next release

Fixed with the latest version (6.2.10)


add user node pool & availability zones

This is a refactory of PR #9

Resync with updated AVM template using grept
Enforce autoscaling ( as per design doc )
Enforce the creation of 3 zonal node pools when the region supports zones.

@lonegunmanb this one is ready to review now. thanks

@nellyk  for future reference this is how I updated with grept:
GITHUB_REPOSITORY_OWNER=Azure GITHUB_REPOSITORY=Azure/avm-ptn-aks-production ~/go/bin/grept apply -a git::https://github.com/Azure/Azure-Verified-Modules-Grept//terraform


Fix user role being removed #2

This "reverts" my own fix #3014
And implements another condition which tests if role_id is null. If so, it will use the old one.
Not sure if its safe to check if slug == users
Tagging @MrCrayon @cyrildewit and @HuntersSerge

This seems like a better solution! 👍

I have to test it but I think this will still remove the additional roles that are hidden in profile editing.

Thats true. Lets see

IMHO there is no way to fix it as it is without some more radical change as in my PR.
Right now with the current UserPolicy I can't change my role even if I can create and change roles and I can edit users and that for me already doesn't make much sense.
Also I don't think it's a good idea to check user_belongstomany_role_relationship in insertUpdateData function because it will leave a specific behavior that might not be needed or wanted extending User and Role models/controllers.

Ok I think this is better now.
I reverted my old changes (belongsToMany can be removed again) and moved data->save() after relationship-handling.
Roles now get reverted when the user-model is saving, when the user-id == data-id and the user can't change his own roles.
@MrCrayon

Also I don't think it's a good idea to check user_belongstomany_role_relationship in insertUpdateData function because it will leave a specific behavior that might not be needed or wanted extending User and Role models/controllers.

What?

Clsoing in favor of #3177


[wl_output] Not seeing scale factor and done events

I only see geometry and mode events.  Scale factor, and done event are not generated.
Example
Using the same compositor (Fedora 33 stock) with Weston C client code, I do get all four events as expected.

You are binding version 1 of the global (the argument given to instantiate_exact), while these events were introduced in version 2.

You are binding version 1 of the global (the argument given to instantiate_exact), while these events were introduced in version 2.

Ah!  Cool, that fixed it.

Ah!  Cool, that fixed it.


does it make sense to block lucid usage if there any errors

There are a few errors that can happen

network error - in case of the user having their wallet connected to a disallowed network
wallet api error - if the user rejected wallet connection
account error - if something unexpected happened when getting account (address) info

All of these should, subjectively, disable blockchain interactions. I think the situations that would cause the two latter of the above, physically makes it impossible to interact with the blockchain, but the first kind of error - network error - would not disallow wallet interactions, but could lead to unexpected, or unwanted, behaviour since the user would be able to interact with the wrong network.
I think disallowing lucid usage (would need to find out how) when operating on the wrong network would be valuable. Would need to find out in what way to prevent it so it makes sense. Either set lucid to undefined (this could cause cascading effects, or moment 22 situations, or somehow else block it, which would be hard to control, since the developer has full access to the lucid instance.

Yes, it does make sense. I have implemented this, and was able to detect a rase condition which made Lucid connect to the wrong network when switching between providers. This does put additional emphises on #8 which will be very important to fully assert before launch.

in short useCardanoContext() now includes an additional property isValid. Atm it only correlates to wether the lucid instance is defined or not, but there are other potential causes that could be handled here in the future.


Allow extended set for  and ignore ref style links, already linked items, and attr_list cases with '#' before the ref

The <num> part of the reference can now match any letter, digit, underscore, or hyphen.
For example:
TAG-xy12_3-z  ==> [TAG-xy12_3-z](http://gh/xy12_3-z)

Any reference preceded by a #, [, or / character is now ignored. This allows ignoring of already linked references, reference style links, and basic use of the pymarkdown attr_list extension. The following are now ignored in the markdown text.
[TAG-123](http://gh/TAG-123)
[TAG-456][tag456]
[TAG-456] [tag456]
## Feature 1 { #F-001 .class-feature }

[tag456]: http://gh/TAG-456

Tests were added for each of these items. Tests and build pass.

Please use if this helps and thank you for this plugin. It has been very useful for some of my documentation projects.

LGTM!


Build dataset/dataloader for multi-parted graph

❓ Questions & Help
Hi, I am working with context-aware datasets for recommendation and I would like to use your library for mini-batching the graph. However, I don't know how to match the mini-batching of my samples (triplets from training data) with GCN mini-batch.
For now, I have a dataset with samples batches of data in the shape of (u, i, c, j, L) -- where u = user / i = positive item / c=context / j = negative item / L = label. I have build a torch.utils.data.Dataset which I named PairData and returns data like:
     [(536, 1157, 2804, 1711, 1.0),
     (453, 1194, 2881, 1864, 1.0),
     (864, 1871, 3036, 2487, 1.0),
     (387, 1450, 2771, 1006, 1.0),
     (341, 1077, 3727, 2123, 1.0),
     (737, 1350, 2631, 2005, 1.0),
     (436, 1230, 2910, 2268, 1.0),
     (806, 1146, 3017, 1989, 1.0),
     (140, 1814, 2952, 1941, 1.0),
     (683, 1307, 2639, 1525, 1.0),
     (234, 943, 2759, 2168, 1.0),
     (644, 1033, 3277, 2312, 1.0),
     (452, 1128, 3276, 1927, 1.0),
     (936, 1239, 2643, 970, 1.0),
     (748, 1029, 4064, 1260, 1.0),
     (88, 1678, 3340, 1273, 1.0)]

CODE:
class PairData(data.Dataset):
def __init__(self, data, sampler=None, adj_mx=None, context=False, is_training=True):
    """
    Dataset formatter adapted pair-wise algorithms
    Parameters
    ----------
    neg_set : List,
    is_training : bool,
    """
    super(PairData, self).__init__()
    self.features_fill = []
    self.context = context
    self.adj_mx = adj_mx  # Sparse coo mx
    self.is_training = is_training
    self.set = data
    self.sampler = sampler

    self._neg_sampling()

def __len__(self):
    return len(self.features_fill)

def _neg_sampling(self):
    if self.is_training:
        neg_set, _ = self.sampler.transform(self.set, is_training=self.is_training, context=self.context, pair_pos=self.adj_mx)
        self.neg_set = neg_set
    else:
        assert self.sampler is None and self.adj_mx is None
        self.neg_set = self.set

    for u, i, c, r, js in self.neg_set:
        if isinstance(c, list):
            u, i, r = int(u), int(i), np.float32(1)
        else:
            u, i, c, r = int(u), int(i), int(c), np.float32(1)
        if self.is_training:
            for j in js:
                self.features_fill.append([u, i, c, j, r])
        else:
            self.features_fill.append([u, i, c, i, r])

def __getitem__(self, idx):

    features = self.features_fill
    user = features[idx][0]
    item_i = features[idx][1]
    context = features[idx][2]
    item_j = features[idx][3]
    label = features[idx][4]
    return user, item_i, context, item_j, label

The Dataset class performs negative sampling inside, thus filling up the 4th column ("j" items) with those items which stands as zero entries in the adjacency matrix. As a model, I use GCN to get embeddings of the nodes. However, I do not know how to perform mini-batch . I don't know if it's possible to perform convolutions just on those nodes that the dataloader is randomly sampling.
My dataLoader looks like:
train_dataset = PairData(train_set, sampler=sampler, adj_mx=adj_mx, is_training=True)

def custom_collate(batch):
    return list(zip(*batch))

train_loader = data.DataLoader(
    train_dataset,
    batch_size=int(args.batch_size),
    shuffle=True,
    num_workers=args.num_workers,
    collate_fn=custom_collate
)

For now, my model is like:
class GCE(torch.nn.Module):
def __init__(self, field_dims, embed_dim, features, train_mat_edges):

    super(GCE, self).__init__()

    self.A = train_mat_edges (edge_index object from PyG)
    self.features = features  # so far, Identity matrix
    self.GCN_module = GCNConv(field_dims, embed_dim, cached=True)

def forward(self, x):
    """
    :param x: Long tensor of size ``(batch_size, num_fields)``
    """
    return self.GCN_module(self.features, self.A)[x]

And then i just define self.embeddings = GCE(max_dim, factors, X, A) and perform matrix factorization in the fordward pass:
class PairMF(nn.Module):

def __init__(self, user_num, max_dim, factors=32, X=None, A=None, dropout=0):
    """
    self.embeddings = GCE(max_dim, factors, X, A, dropout=dropout)

def forward(self, u, i, j, c):
    embeddings_uic = self.embeddings(torch.stack((u, i, c), dim=1))
    embeddings_ujc= self.embeddings(torch.stack((u, j, c), dim=1))

    pred_i = embeddings_uic.prod(dim=1).sum(dim=1)
    pred_j = embeddings_ujc.prod(dim=1).sum(dim=1)
    return pred_i, pred_j

def predict(self, u, i, c):
    pred_i, _ = self.forward(u, i, i, c)

    return pred_i.cpu()

Finally, I perfrom the BPR loss...
I would like to know if there is a way of doing mini-batch with GCN, for example with ClusterData and ClusterLoader, so that I can align my data sampling with GCN sampling, as for now I am fitting the entire graph on memory and, of course, it does not work with bigger data.

========================================================================================
I have looked at all your documentation but I do not know how to deal with my data (is MovieLens dataset, for example) in a useful manner. Maybe the most similar thing could be to build a dataset of similar shape with PUBMED, as you show in this notebook. Thus, I have built the following dataset but I don't know how to follow in:

leave-one-out strategy split

how to mark the n  different types of nodes (in this example n=3 for users, items, context)

how to match GCN sampling with triplets
I'm very confused actually about how to address the problem of building correct dataset... If you could give me some insights about how should I do it it would be nice! This is what I have built:
      class GraphDataset(InMemoryDataset):
      def __init__(self, root, users=None, items=None, transform=None, pre_transform=None):
          super(GraphDataset, self).__init__(root, transform, pre_transform)
          self.data, self.slices = torch.load(self.processed_paths[0])
          self.num_items = self.data.num_users.item()
          self.num_users = self.data.num_items.item()
          self.num_context = 0

      @property
      def raw_file_names(self):
          return [f'u.data']

      @property
      def processed_file_names(self):
          return ['post_processed.pt']

      def download(self):
          # Download to `self.raw_dir`.
          #'curl -O http://files.grouplens.org/datasets/movielens/ml-1m.zip
          #unzip ml-1m.zip'
          pass

      def len(self):
          return len(self.processed_file_names)

      def process(self):
          df = pd.read_csv(os.path.join(self.root, self.raw_file_names[0]), sep='\t', header=None,
                           names=['user', 'item', 'rating', 'timestamp'], engine='python')

          df['rating'] = 1.0
          df['user'] = pd.Categorical(df['user']).codes
          df['item'] = pd.Categorical(df['item']).codes

          user_num = df['user'].nunique()
          item_num = df['item'].nunique()
          df['item'] = df['item'] + user_num
          num_nodes = df['item'].max()

          target_nodes = df['item'].values
          source_nodes = df['user'].values
          edge_index = torch.LongTensor([source_nodes, target_nodes])

          x = torch.LongTensor(np.identity(num_nodes + 1))
          y = torch.FloatTensor(df['rating'].values)

          data = Data(x=x, edge_index=edge_index, y=y)
          data.num_users = user_num
          data.num_items = item_num
          data, slices = self.collate([data])
          torch.save((data, slices), self.processed_paths[0])

In order to load it I do:
    pyg_dataset = GraphDataset(root=f'data/{args.dataset}')
    data = pyg_dataset[0]

My dataset and data look like:
GraphDataset(1)
Data(edge_index=[2, 100000], num_items=1682, num_users=943, x=[2625, 2625], y=[100000])
Then I use ClusterLoader to split the data, and this is where I get stucked:
    cluster_data = ClusterData(data, num_parts=128)  # 1. Create subgraphs.
    train_loader = ClusterLoader(cluster_data, batch_size=32, shuffle=True)  # 2. Stochastic partioning scheme.
    model = PairMF(pyg_dataset, embedding_dim=args.factors, GCE_flag=args.gce, gpuid=args.gpu, dropout=args.dropout)
    # Define loss and optimizer [...]
    model.train()

    for sub_data in train_loader:  # Iterate over each mini-batch.
        sub_data = sub_data.to(device)
        out = model(sub_data)  # Perform a single forward pass.

MODEL
      class GCN(torch.nn.Module):
          def __init__(self, hidden_channels, dataset):
              super(GCN, self).__init__()
              self.conv1 = SGConv(dataset.num_node_features, hidden_channels)

          def forward(self, x, edge_index):
              x = self.conv1(x, edge_index)
              return x

    class PairMF(nn.Module):
        def __init__(self, dataset, embedding_dim=16, GCE_flag=False, gpuid='0', dropout=0):
            super(PairMF, self).__init__()

            self.gcn = GCN(embedding_dim, dataset)

        def forward(self, sub_data):
        # def forward(self, u, i, j, c):
            # pred_i = self._out(u, i, c)
            # pred_j = self._out(u, j, c)
            # return pred_i, pred_j
            return self._out(sub_data)

        def _out(self, sub_data):
            embeddings = self.gcn(sub_data.x.float(), sub_data.edge_index)

            return []

I do not understand how to mixt both concepts...I guess I need to mixt all of this with something like bi-partite graphs but I don't know how to do it...
Thank you very much in advance for your time and for building such a useful library !!

As far as I understand, you want to perform mini-batching on the data returned by PairData, and therefore only want to consider relevant nodes in GCNConv.
The easiest way to achieve this is combining DataLoader and NeighborSampler behaviour. For example, given a returned mini-batch [u, i, j], you can get all relevant node ids via n_id = torch.unique(torch.cat([u, i, j]), and input those to sample their neighbors:
sampler = NeighborSampler(edge_index, sizes=[10, 5])

for data in pair_data_loader:
    ...
    n_id = torch.unique(torch.cat([u, i, j])
    _, n_id2, adjs = sampler.sample(n_id)
    embedding = gnn(x[n_id2, adjs)
    ...

You can find an example of utilizing NeighborSampler here.

Hi,
Thanks for your time!
Actually, what I want is on one side, to perform the convolution based on u -i- c (user interacted with an item on a given context), which are 3 different types of nodes connected. On the other side I'll perform the convolution on u -j -c (where represents that a user did NOT interact with "j" item on that context). But to perform the second one is the same as the first one being like:
        def forward(self, u, i, j, c):
            pred_i = self._out(u, i, c)
            pred_j = self._out(u, j, c)

            return pred_i, pred_j

However, I have defined the sampler as sampler = NeighborSampler(edge_index, sizes=[10, 5]), and I have looked at your example as well, but I do not understand what does the sampler return.
    n_id = torch.unique(torch.cat([u, i, c]))

len(n_id)  == 101, which is taken all the different unique nodes that are important in this mini-batch convolution

    In [28]: len(n_id2)
    Out[28]: 2031

    In [29]: len(adjs)
    Out[29]: 2

I do not get either what are the different edge_index representing:
    In [30]: adjs[0].edge_index
    Out[30]:
    tensor([[ 737,  738,  597,  ..., 2029, 1593, 2030],
            [   0,    0,    0,  ...,  736,  736,  736]])

    In [31]: adjs[1].edge_index
    Out[31]:
    tensor([[101, 102, 103,  ...,  10,  35, 736],
            [  0,   0,   0,  ..., 100, 100, 100]])

Thank you again for your time!

For more information, normally I would do this:
MODEL:
class GCE(torch.nn.Module):
def init(self, field_dims, embed_dim, features, train_mat_edges, dropout=0, args=None, gce_loader=None):
            super(GCE, self).__init__()

            self.A = train_mat_edges
            self.features = features  # so far, Identity matrix
          self.GCN_module = GCNConv(field_dims, embed_dim, cached=True)

        def forward(self, x):
            """
            :param x: Long tensor of size ``(batch_size, num_fields)``
            """
            return self.GCN_module(self.features, self.A)[x]

PairMF model:
     [...]
     self.embeddings = GCE(max_dim, factors, X, A, dropout=dropout, args=args)\

    def _out(self, u, i, c):
        self.embeddings(torch.stack((u, i, c), dim=1)).   # shape [bs, 3, embedding_size])

I would like to obtain the same "self.embeddings" but just feeding to the GCN the (u,i,c) from the mini-batch. Thanks!

The NeighborSampler returns node indices which represent all nodes involved in the current computation graph, and a list of edge indices which represent the computation graph (forming a tree-like structure), where, finally, you obtain the node embeddings for the initial n_id passed to the sampler.
In my example, I unified positive and negative examples into a single n_id vector, but you can also distinguish those by calling sampler.sample() twice and perform two model calls, e.g.:
n_id, adjs = sampler.sample(i)
pos_embedding = gnn(x[n_id], adjs)

n_id, adjs = sampler.sample(j)
neg_embedding = gnn(x[n_id], adjs)

Let me know if that helps you :)

Hi,
Thank you a lot for your time. Though, I'm afraid I am still a bit confuse...
So, going back to the example where I have u, i and c variables which refers to user, item and context nodes respectively.
In [14]: unique_uic
Out[14]:
tensor([ 820,  977, 1007, 1013, 1016, 1019, 1029, 1052, 1066, 1085, 1099, 1101,
1107, 1113, 1114, 1121, 1160, 1173, 1183, 1184, 1187, 1192, 1220, 1228,
1244, 1275, 1280, 1304, 1308, 1314, 1324, 1329, 1362, 1392, 1403, 1405,
1410, 1450, 1472, 1483, 1505, 1516, 1522, 1550, 1565, 1567, 1573, 1574,
1589, 1600, 1603, 1624, 1633, 1641, 1643, 1645, 1722, 1741, 1750, 1773,
1777, 1807, 1856, 1873, 1888, 1906, 1910, 1920, 1921, 1963, 1984, 2005,
2058, 2067, 2090, 2164, 2187, 2226, 2232, 2240, 2260, 2289, 2319, 2324,
2331, 2356, 2396, 2448, 2456, 2459, 2467, 2481, 2500, 2522, 2523, 2535,
2561, 2562, 2591, 2610, 2694], device='cuda:0')`
Then, I define the NeighbourSampler and I get the n_id and the adjs: _, n_id, adjs = self.sampler.sample(unique_uic.cpu()) and I get:
In [15]: n_id Out[15]: tensor([ 820,  977, 1007,  ..., 3065,   19, 2873])
In [16]: adjs
Out[16]:
[Adj(edge_index=tensor([[   0,    2,    2,  ..., 2046, 2047, 2048],
[ 105,  296,  352,  ...,  741,  744,  746]]), e_id=tensor([  5078, 340087, 149833,  ...,
167333, 167976, 319386]), size=(2049, 747)),
Adj(edge_index=tensor([[100, 101, 102,  ..., 744, 745, 746],
[ 37,   0,   0,  ..., 100, 100, 100]]), e_id=tensor([245579,   5199,   5064, 309849,   42
72,   5118,   5079,   5160,   5051, ..., 59842, 319712]), size=(747, 101))]
But here is where I get stucked. I understand what that n_id represents node indices which are all nodes involved in the current computation graph and for which I will get the corresponding embeddings as GCN result but I do not get how should I enter adjs variable in GCN layer... as it is a list of two Adj() instances.
The only way I have found to make the model work is by entering:
X = self.features.to_dense()[n_id] A =  adjs[0].edge_index self.GCN(X, A)
But however, I do not understand either If I should enter adjs[0] or adjs[1], neither what does adjs host.
Thank you so much again for your time!
PS: I'll be faster answering now, was handling other stuff this month, sorry for that!

adjs is a list that holds the edge connectivity information per layer. You will need to use different adjacency matrices in each layer, similar to this example.

Thank you very much!
I just have one last question:
When I pass to the model the unique idx so that it computes the corresponding embeddings, following the example, I pass the variable (whose  len(unique_uic) == 101) :
`In [32]: unique_uic
Out[32]:
tensor([ 820,  946,  959, 1009, 1010, 1013, 1044, 1109, 1147, 1149, 1152, 1182,
        1184, 1208, 1222, 1267, 1308, 1334, 1409, 1417, 1421, 1431, 1455, 1457,
        1462, 1470, 1475, 1565, 1578, 1586, 1628, 1632, 1675, 1686, 1690, 1698,
        1723, 1733, 1737, 1745, 1775, 1802, 1817, 1835, 1854, 1860, 1864, 1868,
        1879, 1892, 1894, 1896, 1915, 1918, 1930, 1947, 1951, 1968, 1978, 1984,
        2037, 2042, 2045, 2070, 2075, 2086, 2106, 2125, 2186, 2199, 2205, 2290,
        2302, 2303, 2332, 2338, 2347, 2363, 2368, 2381, 2382, 2417, 2440, 2441,
        2449, 2464, 2466, 2471, 2500, 2512, 2519, 2548, 2549, 2557, 2568, 2578,
        2580, 2582, 2617, 2618, 2694])
`

However, when I apply the sampler like _, n_id, adjs = self.sampler.sample(unique_uic.cpu()), the n_id variable has the lenght of 2000. I understand that the first 101 are the ones that i want and the others are other nodes involved in the convolution, right? If that's okay, then it would be okay to end up getting the embeddings like:
`embeddings_uic = self.embeddings(n_id, adjs)[:len(unique_uic)]`

However, then I end up with 101 embeddings which I need to keep in the original stacked form in order to feed them to Factorization Machines model to perform pairwise operation. It requires the following stacked form:
`Out[34]:
tensor([[ 820, 2568, 2694],
        [ 820, 1802, 2694],
        [ 820, 2578, 2694],
        [ 820, 2580, 2694],
        [ 820, 1044, 2694],
        [ 820, 2070, 2694],
        [ 820, 2582, 2694],
        [ 820, 1817, 2694],
        [ 820, 2075, 2694],
                  .........
        [820, 2042, 2694],
        [ 820, 2045, 2694],
        [ 820, 2302, 2694],
        [ 820, 2303, 2694]])
`

It should have the shape [99, 3] as it refers to the 99 interactions (in this example) of user-item-context. However, when i get the embeddings i just get those 101 corresponding to the unique id's. Is there a way to keep the mapping of the node number that we inputed and the resulting embedding?
Thank you very much!

The self.embeddings module should already return a tensor of shape [101, ...]. I'm not yet sure I understand, but for the mapping, unique_uic already contains the mapping from global to local indices.

Hi,
Thanks, I'll just do a mapping to get the embeddings stacked again in the original shape of U-I-C.
I'll leave you here the paper of our work in case you want to take a look. I can also share my code in case you find it interesting.

Hi,
I realized that edge_index is not giving me what I really want. As I explained, I have three different type of nodes and, for example, they are not connected among the same type nodes. So, Following with the example i have users from 0 to 943, then items from 944 to 2626 and the rest of nodes as context. So, If i do what we said:
`In [3]: n_id
Out[3]: tensor([ 820,  954,  955,  ..., 2392, 2741,   24])
In [4]:         edge_index, _, size = adjs[0]  # first layer A
In [5]: edge_index
Out[5]:
tensor([[   1,    1,    1,  ..., 2055, 2056, 2057],
[ 122,  361,  486,  ...,  734,  734,  735]])`
I actually don't think its okey because in the edge_index variable there are connections among same type of nodes (1-122, 1-361, etc). Would it be possible to do like a n-partite graph with mini-batch?
I'll leave you here the paper of our work in case you want to take a look. I can also share my code in case you find it interesting.
Thanks

Note that the NeighborSampler will perform a relabeling of nodes, so that subgraphs have consecutive indices from 0 to N-1. You can get the original graph indices via
global_sub_edge_index = n_id[edge_index]

Okey, thanks!
Now, sometimes I get the following error. Do you know what am I missing?
` Traceback (most recent call last):                                                           [18/1833]

File "main.py", line 468, in
main()
File "main.py", line 447, in main
desc=total_info)
File "/home/paulagomez/GCE/model/pair/train.py", line 39, in train
tune=tune, populary_dict=None)
File "/home/paulagomez/GCE/utils/splitter.py", line 46, in perform_evaluation
prediction = model.predict(user_u, item_i, context)
File "/home/paulagomez/GCE/model/pair/FMRecommender.py", line 117, in predict
pred_i, _ = self.forward(u, i, i, c, inference=True)
File "/home/paulagomez/GCE/model/pair/FMRecommender.py", line 59, in forward
pred_i = self._out(u, i, c)
File "/home/paulagomez/GCE/model/pair/FMRecommender.py", line 86, in _out
embeddings_ui = self.embeddings(n_id, adjs)[:len(unique_uic)]
File "/home/paulagomez/.virtualenvs/research/lib/python3.6/site-packages/torch/nn/modules/module.py"
, line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/paulagomez/GCE/model/GCE/gce.py", line 91, in forward                    [0/1833]
return self.GCN_module(self.features[ids], edge_index)
File "/home/paulagomez/.virtualenvs/research/lib/python3.6/site-packages/torch/nn/modules/module.py"
, line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/paulagomez/.virtualenvs/research/lib/python3.6/site-packages/torch_geometric/nn/conv/gcn
_conv.py", line 169, in forward
size=None)
File "/home/paulagomez/.virtualenvs/research/lib/python3.6/site-packages/torch_geometric/nn/conv/mes
sage_passing.py", line 233, in propagate
kwargs)
File "/home/paulagomez/.virtualenvs/research/lib/python3.6/site-packages/torch_geometric/nn/conv/mes
sage_passing.py", line 158, in collect
j if arg[-2:] == '_j' else i)
File "/home/paulagomez/.virtualenvs/research/lib/python3.6/site-packages/torch_geometric/nn/conv/mes
sage_passing.py", line 127, in lift
return src.index_select(self.node_dim, index)
IndexError: index out of range in self`
thank you again!

You still need to input the edge_index into the GNN returned by NeighborSampler. The aforementioned assignment is just a way to get the original graph structure back.


Disallow null for optional strings

When updating from version 4.0.1 to 5.0.0-rc2 we encounter the following problem: String properties that are NOT marked as Required are suddenly created with "Disallow null". I think it is a big problem that is not possible to have nullable string properties or is there another way to achieve this in v5?

Please provide exact repro steps

I have attached an example solution, see GetCorrespondenceAddress in the ValuesController:
OptionalStringsNotNullable.zip
The CorrespondenceAddress has one required string property (Street) and two optional properties ( HouseNumber, HouseNumberExtension):
public class CorrespondenceAddress
{
    [Required]
    public string Street { get; set; }

    public int? HouseNumber { get; set; }

    public string HouseNumberExtension { get; set; }
}

In the swagger.json, the following is created:
"schemas": {
      "CorrespondenceAddress": {
        "required": [
          "street"
        ],
        "type": "object",
        "properties": {
          "street": {
            "type": "string"
          },
          "houseNumber": {
            "type": "integer",
            "format": "int32",
            "nullable": true
          },
          "houseNumberExtension": {
            "type": "string"
          }
        },
        "additionalProperties": false
      }

The houseNumber is nullable as expected, but the houseNumberExtension is not.
We then use nswag to create a client (the .nswag file and the resulting generated code are also included in the example project).
The CorrespondenceAddress class in the client is created as follows:
[System.CodeDom.Compiler.GeneratedCode("NJsonSchema", "<IP_ADDRESS> (Newtonsoft.Json v<IP_ADDRESS>)")]
public partial class CorrespondenceAddress
{
    [Newtonsoft.Json.JsonProperty("street", Required = Newtonsoft.Json.Required.Always)]
    [System.ComponentModel.DataAnnotations.Required(AllowEmptyStrings = true)]
    public string Street { get; set; }

    [Newtonsoft.Json.JsonProperty("houseNumber", Required = Newtonsoft.Json.Required.Default, NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore)]
    public int? HouseNumber { get; set; }

    [Newtonsoft.Json.JsonProperty("houseNumberExtension", Required = Newtonsoft.Json.Required.DisallowNull, NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore)]
    public string HouseNumberExtension { get; set; }
}

However, I would expect that the HouseNumberExtension has Required.Default, just as the HouseNumber.

This should be fixed by 0f7f6e3fa47c76bd418b2ea5e9d1683e54562e84
With that said, it's worth noting that, with the OpenAPI 3 (OA3) specification, there's a difference between required and non-nullable. A required property MUST be provided in the payload but it's value MAY be null, as indicated by the nullable property in the spec. So, with the latest change, the following schema definition will be generated:
"schemas": {
      "CorrespondenceAddress": {
        "required": [
          "street"
        ],
        "type": "object",
        "properties": {
          "street": {
            "type": "string",
            "nullable": true
          },
          "houseNumber": {
            "type": "integer",
            "format": "int32",
            "nullable": true
          },
          "houseNumberExtension": {
            "type": "string",
            "nullable": true
          }
        },
        "additionalProperties": false
      }

@tmacht please let me know if this makes sense?

@domaindrivendev, is 0f7f6e3 already in 5.0.0-rc3-preview-0954 ? I createad equal test case but "nullable" was not generated to json definition.

@mkontula - perhaps you could confirm but I think now the issue is a little different? With preview-0954 nullable: true is correctly being emitted for the string properties but not for the nullable int? At least that's what I'm seeing when I try to repro.
Either way, it's an issue that needs to be addressed before the official Nuget goes out. Re-opening and assigning to the v5 milestone.

I can confirm, preview-0954, nullable: true is being emitted to string properties.
Testing with the class below, nullable int and datetime value types did not get the nullable: true
public class ValueModelDto
{
    [Required]
    public int Id { get; set; }
    [Required]
    public string RequiredString { get; set; }
    public string OptionalString { get; set; }
    [Required]
    public int RequiredInt { get; set; }
    public int? OptionalInt { get; set; }
    [Required]
    public DateTime RequiredDate { get; set; }
    public DateTime? OptionalDate { get; set; }
}

Should be fixed by 24d166d804b677d00747428c2561744e682d10e7 and available for test in the latest preview nuget on myget.org (5.0.0-rc3-preview-0959 at time of writing)

Nice! Thanks for the fix. Works like a charm!


Update doc navigation to handle more content

Our Documentation left-side navigation is getting a bit long and we have a desire to add more pages and sections which would make it even longer. We also only support one level of nesting (Section > Page) and new content might be better grouped with another level.
With the current design we chose not to made the sections collapsible because we wanted the nav to be easily scannable. This works will for the above-the-fold sections, but the nav is so long now that it requires a lot of scrolling to even see what the sections are.
One way of handling this would be to have section headers be clickable and collapsible, but with some nuance. Not all sections have to be collapsible, we have some space to play with on the headers: (collapsible, not collapsible) x (default open, default closed). We can set the default expanded state depending on the importance  of the section. For instance we can keep Introduction, Components, and Templates expanded and default to collapsed for everything else. If we have multi-level nesting, we can also default to expanded for the top-level section and collapsed for sub-sections
▼ INTRODUCTION
    What is Lit?
    Getting Started

▼ COMPONENTS
    Overview
    Defining
    Rendering
    Reactive properties
    Styles
    Lifecycle
    Shadow DOM
    Events
    Decorators

▼ TEMPLATES
    Overview
    Expressions
    Conditionals
    Lists
    Built-in directives
    Custom directives

▶ COMPOSITION

▼ USE CASES
  ▶ STANDALONE COMPONENTS
  ▶ APPLICATIONS
  ▶ DESIGN SYSTEMS

▶ FRAMEWORK INTEGRATION

▶ TOOLS AND WORKFLOWS

One problem with collapsible sections is keeping the expanded/collapsed state across page navigations. We can remember this state in local storage or session storage, though that state leaks between tabs. It might be the best option. Another is to record state in the URL so that it's local to a tab, but this can effect cachability of the page if the state is in query parameters, or the simplicity of native fragment navigation if it's encoded in fragments.
For cold navigations to URLs within a collapsed section, we must open the section to show the user where they are in the content.

Addressed in #944 and #945 w/ team feedback:
If navigating to first three sections, expand all three of those sections. Otherwise collapse the first three sections


The python version should be 3.x ?

Hi, due to that the division operation is different between python 2.x and python 3.x, there is some
incorrectly code of  division operation if python 2.x is used. For example, line 107 of GConv.py, the
theta would be 0 for all filter. Hope you could mention the python version in README.

@gzhcv Thanks for you kind advise. Since I am quite busy these days, I will fix this issue later.

@jxgu1016  Thanks! I just no sure if this is a issue,  because i visualized the filter in GoFs and find it seems each filter is different with others.

For the filters of first layer,  in the beginning training stage, the filters obtained by modulating a learned convolutional filter using different orientation Gabor filters are different with each other.

But after training for 0.x million steps ,  the filters are quite similar to each other (see the following figure), but actually the values are different.

For the filters of second layer,  the filters are different to each other even training for  0.x million steps.

So, it seems this is not a issue. I just not very clear about how the GConv.py is used for build the Python package.

Thanks for your detailed report,  and I will check it later.

@gzhcv My python version is 2.7.  There should be some 0 in the filters. It  seems not to be a issue. Thx for your report anyway.

@jxgu1016  The parameter theta  represents different orientation of Gabor filter, if the GoF is obtained by the code in GConv.py (since theta  is always be 0 that computed in line 107 of GConv.py), why the learned filters are different with each other?  I'm confused about this.

@gzhcv you may miss the codes on the 1st line: from __future__ import division.

@jxgu1016 Oh....  Sorry for my carelessness.


Core: Research JetStream

Action Item
Research and document in this ticket:

[ ] How to switch to JetStream
[ ] Should we change any configuration for its deployment?
[ ] Is there any breaking change for other services?
[ ] Compared to NATS, how more/fewer resources are needed?

Reasearch findings:
In General, JetStream dstinguishes between Push-based and Pull based subscribers:

When creating a push-based subscription, the subscriber will receive all events for a certain topic, including the ones that have happened in the past. The main use case for these kinds of subscribers is for logging/monitoring services that need to derive a state from all previous events, but do not have to execute any tasks or produce results.
Pull based subscribers can be used for implementing an exactly-once message delivery system. These subscribers can subscribe to a certain topic by attaching themselves to a Consumer of a stream. A consumer can have a durable name, which means that it will only receive messages that have not been acknowledged by another subscriber attached to that consumer (i.e. at-most once). When a subscriber for a consumer recovers from a downtime, it will receive all unacknowledged messages that have been sent in the meantime since those will be persisted by Jetstream. In our case, a durable name would correspond to a combination of a service name and a topic this service is subscribed to. E.g.
shipyard-controller-sh-keptn-> or helm-service-sh-keptn-event-deployment-triggered. Multiple instances of the same services would then attach themselves to the same consumer.

From an architectural point of view, some adaptations would need to be made, since in JetStream, the streams, as well as the consumers have to be managed. In order for the eventing to work, a stream containing all topics of the Keptn event domain needs to be set up. With our current event topic names, the stream would simply be called sh - the name of a stream must start with the common prefix that all topics share, and must not contain a . - therefore, sh.keptn can unfortunately not be used as a stream name - Simply calling it sh seems a bit hacky though.
Also, ideally the consumers would also be managed by a central component based on the available topics and integrations within the uniform.
Both the streams and consumers can in theory be also created as needed by the distributors, but this would likely lead to problems regarding race conditions and in general is not a clean solution.

Thank you @bacherfl for your research! I will close the ticket since all points have been addressed.


Implement 3 options for the wallet login (existing, PIN based, no PIN)

middleware in APIHub named  "secrets", persisting secrets for SSO logedin users

endpoints

GET /getSecret/userId/appName/

PUT /putSecret//userId/appName/

For login mode "pin" and "ssodirect" WalletDB is created using a random secret (256 bits) associated with currently logged userId  and for the current wallet type (appName)

new modes are: sso-direct and sso-pin


Export-TfsWorkItemAttachment - 'No filter arguments have been specified'

Export-TfsWorkItemAttachment -Destination "$DestFolder" -Force -WorkItem $CSO.id -Attachment "$PDF" -Collection $coll
Export-TfsWorkItemAttachment : No filter arguments have been specified. Unable to perform a simple query.
At line:1 char:1

Export-TfsWorkItemAttachment -Destination "$DestFolder" -Force -WorkI ...

  + CategoryInfo          : NotSpecified: (:) [Export-TfsWorkItemAttachment], ArgumentException
  + FullyQualifiedErrorId : System.ArgumentException,TfsCmdlets.Cmdlets.WorkItem.Linking.ExportWorkItemAttachment

Export-TfsWorkItemAttachment -Destination "$DestFolder" -Force -WorkItem $CSO.id -Attachment "$PDF" -Collection $coll -verbose
VERBOSE: [14:25:28.7200] [Export-TfsWorkItemAttachment] Running cmdlet with parameter set '__AllParameterSets' and the following arguments:
Export-TfsWorkItemAttachment : Self referencing loop detected with type 'Microsoft.TeamFoundation.Framework.Client.CatalogNode'. Path
'Collection.CatalogNode.Resource.NodeReferences'.
At line:1 char:1

Export-TfsWorkItemAttachment -Destination "$DestFolder" -Force -WorkI ...

  + CategoryInfo          : NotSpecified: (:) [Export-TfsWorkItemAttachment], JsonSerializationException
  + FullyQualifiedErrorId : Newtonsoft.Json.JsonSerializationException,TfsCmdlets.Cmdlets.WorkItem.Linking.ExportWorkItemAttachment

NOTE: This works if I specify the work-item OBJECT, but fails when I try to use the $Item.ID
HELP says:
-WorkItem
Specifies a work item. Valid values are the work item ID or an instance of Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models.WorkItem.

Can you please check the type of the property $CSO.Id? I can reproduce your error when Id is not an Int32, as in such cases the cmdlet doesn't know how to properly parse it.
One way to work around the issue (assuming that's the case) would be to force a cast, such as in:
Export-TfsWorkItemAttachment -Destination "$DestFolder" -Force -WorkItem ([int] $CSO.id) -Attachment "$PDF" -Collection $coll

right. For some reason it's a string.
$TGTs = $CSOs.where({$_.Relations.LinkType -eq 'Attachment'})
$CSO = $TGTs[33]
$CSO.Id
2521
$CSO.Id.gettype().fullname
System.String

Thanks for confirming, @lukeb1961. I assume that unblocked you.
Although that's a rather edgy case, I'm adding logic to Get-TfsWorkItem so that IDs passed as strings get properly parsed and recognized.
It's coming in the next release. In the meantime, please keep insuring your IDs are passed as ints.
Please keep the feedback coming!


Add 'Babel ES6 JavaScript' as a grammar that supports JSX.

What does this implement/fix? Explain your changes.
Adds language-babel's grammar name to JSX supported languages.
...
Does this close any currently open issues?
No
...
Any other comments?
...
Checklist
Check all those that are applicable and complete.

[ ] Merged with latest master branch
[ ] Added examples for testing to examples/ directory
[ ] Travis CI passes (Mac support)
[ ] AppVeyor passes (Windows support)

Add support for the language-babel package.

Why should this grammar be associated with JSX opposed to JavaScript?  Does this directory support XML tags as language tokens?

The language grammar supports Facebook's JSX, which I believe this language file relates to.

Maybe I've misunderstood how these language files work. I'll close the PR. Thanks

@gandm I recommend reopening the pull request.  You code change is spot on and it sounds like the request is completely accurate if this Babel variant supports JSX additions.  Perhaps you misunderstood because I used the word directory when I meant to use the word directly.  Sorry, its a busy day at work.

@prettydiff Thanks for the info. My package, language-babel, is the one recommended in Facebook's nuclide project, as well as github's jsx highlighter, and I thought that adding it here would help some people who might use this package. Thinking about it, as I also support flow typing, it is probably not appropriate to fix it to a single beautifier.
Thanks again.


I am getting Twitter accounts that I am not following

subject

Could you please provide some additional information?


Update README.md

Remove duplicated Back to Top on question 82, and extended question 87 to remove one question 88 (There were 2 questions 88).

Thanks for PR


Make subcrates #![no_std]

[ ] num-traits
[ ] num-integer
[ ] num-complex
[ ] num-iter

I noticed in the 1.11 release notes that to_degrees and to_radians were just now added to libcore.  Things like this will constrain our rust version for going no_std.  (But when we do make the breaking transition, we'll be choosing a new rust base anyway.)

Any timeline on when this will be done? Especially num-traits?

I'm interested in this too.
to_radians and to_degrees can be implemented easily without Rust support. It's simply self * PI / 180.0 and self * 180.0 / PI (with appropriate conversion).

I started working on porting num-traits to no_std. However, I run into a very irritating problem:
error[E0308]: mismatched types
   --> src/sign.rs:116:38
    |
116 |                 <$t>::abs_sub(*self, *other)
    |                                      ^^^^^^ expected &f64, found f64
...
141 | signed_float_impl!(f64, f64::NAN, f64::INFINITY, f64::NEG_INFINITY);
    | -------------------------------------------------------------------- in this macro invocation
    |
    = note: expected type `&f64`
               found type `f64`
    = help: try with `&*other`

Removing the * does not work:
error[E0308]: mismatched types
   --> src/sign.rs:116:38
    |
116 |                 <$t>::abs_sub(*self, other)
    |                                      ^^^^^ expected f64, found &f64
...
141 | signed_float_impl!(f64, f64::NAN, f64::INFINITY, f64::NEG_INFINITY);
    | -------------------------------------------------------------------- in this macro invocation
    |
    = note: expected type `f64`
               found type `&f64`

So the compiler expects f64 and &f64 in the same place?

Even fixing that problem by reimplementing abs_sub it still does not work:
error: no method named `is_finite` found for type `f64` in the current scope
   --> src/cast.rs:231:19
    |
231 |             if !n.is_finite() || (-max_value as f64 <= n && n <= max_value as f64) {
    |                   ^^^^^^^^^
...
273 | impl_to_primitive_float!(f32);
    | ------------------------------ in this macro invocation
    |
    = help: items from traits can only be used if the trait is in scope; the following traits are implemented but not in scope, perhaps add a `use` for one of them:
    = help: candidate #1: `use float::Float;`
    = help: candidate #2: `use core::num::Float;`

core::num::Float is not stable and float::Float seems to cause an infinite recursion.

Maybe .borrow() could help? (Just a guess, didn't see entire code)

I backported is_finite from core::num::Float. It is just a few lines and does not use unstable intrinsics. However, now I'm getting infinite recursions in float.rs because it seems like almost all math functions are not implemented in core. This means we would have to depend on another crate to implement them.

I think the best way forward is to split Float into the parts that require std and the parts that don't.

Given that #[lang = "f32"] and #[lang = "f64"] are in libstd, I'm tempted to say that we should just chop all floating point from num #![no_std].  But if you can reasonably hang on to some core-only pieces, then let's see what remains...

But if you can reasonably hang on to some core-only pieces, then let's see what remains...

I think we should support everything in this file and the basic operations (Add, Sub, Mul, ...). That would be enough for e.g. sorting floats.

I think it's fine to exclude core's unstable Float methods.  Those are basically all the methods that are lang items in std, so we don't need to support them in no_std.  That still leaves the constants and basic ops.

Unfortunately they are required for sorting floats, it would be nice to
still have is_nan, is_finite etc. They can be implemented in one line
on stable using core.
On Tue, May 30, 2017, 19:32 Josh Stone<EMAIL_ADDRESS>wrote:

I think it's fine to exclude core's unstable Float methods. Those are
basically all the methods that are lang items in std, so we don't need to
support them in no_std. That still leaves the constants and basic ops.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/rust-num/num/issues/216#issuecomment-304951532, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AACCtKsEuKDM9H5P6GDTalvSgtG9tKcfks5r_FLBgaJpZM4JgF8S
.

Closing in favor of the separate issues:

rust-num/num-traits#16
rust-num/num-integer#4
rust-num/num-iter#1
rust-num/num-complex#6


Unable to "Heroku Login" from git

I am unable to login to heroku from git and my browser anymore. This is what I see.
Error:       An existing connection was forcibly closed by the remote host. - SSL_connect (Errno::ECONNRESET)
Command:     heroku logs
Version:     heroku/toolbelt/3.41.3 (i386-mingw32) ruby/1.9.3
Error ID:    8a9df87b2cdc4d4fa3ec76cc8bf64cfb
how do i fix this?

There probably isn't anything I can do to help—this is an issue with your internet connection. If you're behind a proxy make sure it's setup correctly. https://devcenter.heroku.com/articles/using-the-cli#using-an-http-proxy

hey @dickeyxxx thanks a lot for the link. It works 100%. https://devcenter.heroku.com/articles/using-the-cli#using-an-http-proxy


Current released version of rom-csv depends on rom ~> 0.9.0

This makes rom-csv impossible to use alongside rom-sql when installing via rubygems.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/30986758-current-released-version-of-rom-csv-depends-on-rom-0-9-0?utm_campaign=plugin&utm_content=tracker%2F10396075&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F10396075&utm_medium=issues&utm_source=github).

This gem has no official maintainer yet.

We use it in our project, @wafcio can you update deps?

sure, I will do it today later

It's already got the right set of dependencies; just needs to be released.

released on rubygems https://rubygems.org/gems/rom-csv/versions/0.3.0


Where can we get the motion lora map files?

I saw the addition to the readme and would like to try adding them. Can you post a URL?

official repo
https://github.com/guoyww/animatediff/

Got it thanks


🚧Setup GL callstack feature to only work for Debug builds

I have done the items below . . .

[X] I have updated the title by replacing the '<title>' section.

Description
The ability to keep track of the call stack of OpenGL calls has been added to the GLInvoker class.  Currently this works for all builds available, but we only want it to run for Debug builds for performance reason when doing Release builds.
Add a preprocessor directive to the GLInvoker.AddToGLCallStack() method that will only contain code to queue stack information when in Debug mode.
For Release builds, fill the method with a comment that explains that this only is turned on in debug mode.  This way if the library is decompiled, it can explain why the method is empty.
Acceptance Criteria
This issue is finished when:

[ ] Code documentation added if required
[ ] Unit tests added
[ ] All unit tests pass

ToDo Items

[ ] Draft pull request created and linked to this issue
[X] Priority label added to issue (low priority, medium priority, or high priority)
[X] Issue linked to the proper project
[X] Issue linked to proper milestone
[ ] Unit tests have been written and/or adjusted for code additions or changes
[ ] All unit tests pass

Issue Dependencies
No response
Related Work
No response

Groomed


[#3589] Remove the cache from the controller

So this is a follow up fix for #3589? :)

Yes :). The solution of that issue resulted in not using the cache at all, but the code remained.


Alternate debug connection than wifi on real device

Description
I have specific cases I need to test often, where the network should not be available. Sometimes no wifi, sometimes no cellular, sometimes no Bluetooth. Currently, for the case of no wifi or airplane mode, I just bundle the js part, and try to debug by console.loging. This of course is not great or fast.
Solution
So I thought we might be able to do the same debugging but using Bluetooth connection to the MacBook. I guess that would not be too challenging, but I would like some input for that idea. I mean that we could actually choose wifi or Bluetooth by commenting some line of code or something. so we can have the default way, and change it on demand. I'm sure Bluetooth would be useful for people sound rn dev on the go, for example.
One other way which I don't know if it's even possible is to debug with chrome but debug the bundled js. provided that we can bundle the js unminified.
What do you think? is there a way o go around that? is there a way to actually fix that? thanks.
Additional Information

React Native version: 0.43.0
Platform:  iOS
Development Operating System: MacOS
Dev tools: Xcode

I would also love this feature. Often in my location I can be without wifi (either it goes down, or I have to hotspot). When i'm using my phone as a hotspot I want to be able to debug my application.

Have come back round to wanting this functionality again and doing a small bit of digging. I think if we change this file:
Libraries/Core/Devtools/setupDevtools.js
to somehow use this package: https://www.npmjs.com/package/bluetooth-websocket-api
I think we could achieve this, although I'm not entirely sure what i'm doing :/

Is there not a was to use usb for debugging?
I would like to have the debugger working while the device has no wireless connection.

Should there be a post at https://react-native.canny.io/feature-requests?

This would be a great feature, or even over LAN without an Internet connection. Why does the packager require access to the internet if all the node modules are cached locally?


ReadLabelFile is missing a 'return'

During the latest release, ReadLabelFile() in dataset_utils.py was made deprecated. I suspect, the function is now supposed to call read_label_file() but the return statement is missing, so it always returns None. This breaks existing code.
Offending code line

@Martin-Oehler you are correct, good catch!

@Martin-Oehler Update: We've fixed this internally and will release on next update, thanks again!


Add Bengali translation

With approximately 228 million native speakers and another 37 million as second language speakers, Bengali is the fifth most-spoken native language and the seventh most spoken language by total number of speakers in the world. Wiki
So, I've planned to translate the articles and guides in Bengali, my mother tongue.

CONTRIBUTING.md

terjemahan Indonesia bagaimana?

No, it's Bangla, also called Bengali. Mostly used in Bangladesh and West Bengal (Kolkata), India.

@nowshad-hasan, how's the translation going on? If possible, I wanna join as it's been ages but no progress record is found.

Feel free to open a pull request with work in progress at any time. Instructions at https://github.com/github/opensource.guide/blob/master/docs/translations.md

@mlinksva If it overlaps with the work of @nowshad-hasan ? Isn't that be rude or something unappreciated in OSS community?
I'm totally new to this contribution thing.

Hi @fuad021, It's a great pleasure to meet you. I opened this issue but couldn't manage to work on. And it's a big work to do. So, I think we can split up this work. You can translate these particular parts I've mentioned below:

How to Contribute to Open Source
Starting an Open Source Project
Finding Users for Your Project
Building Welcoming Communities
Best Practices for Maintainers

I'll do the rest and try my best to finish within this month.

Closing this in favour of a PR. Looking forward to it!


[CFE-475] Add code snippets for components

This PR adds code snippets/classes (where applicable) for elements and components of the style guide.
Other notable changes:

Refactored icon rendering (render the template using an array instead of repeating HTML with different classes)

Removed text preview function as per new Style Guide design

Merged SVG and Sprite Icons into a unified section

Display SVG Icons within their respective categories

Add syntax highlighting, together with a copy code button for easier development

Todo:

~Add code snippets for video section~
~Remove trailing whitespace~

Would prefer to merge this before https://github.com/luxola/sephora-style-guide/pull/71 as the sidebar has to reference the new sections added in this PR.


Installing GEOS on windows

I am trying to install geos on windows 10 using cygwin. When make command is given, it shows:
make: *** No targets specified and no makefile found.  Stop.
I am unable to install in on my system so I tried it with pip. The pip installed geos, but basemap can not find the path of 'GEOS_DIR`.
How can I download basemap?

The python setup.py install for basemap gives the following error:
checking for GEOS lib in C:\Users\lsrathore ....
checking for GEOS lib in C:\Users\lsrathore\local ....
checking for GEOS lib in /usr ....
checking for GEOS lib in /usr/local ....
checking for GEOS lib in /sw ....
checking for GEOS lib in /opt ....
checking for GEOS lib in /opt/local ....

Can't find geos library in standard locations ('C:\Users\lsrathore', 'C:\Users\lsrathore\local', '/usr', '/usr/local', '/sw', '/opt', '/opt/local').
Please install the corresponding packages using your
systems software management system (e.g. for Debian Linux do:
'apt-get install libgeos-3.3.3 libgeos-c1 libgeos-dev' and/or
set the environment variable GEOS_DIR to point to the location
where geos is installed (for example, if geos_c.h
is in /usr/local/include, and libgeos_c is in /usr/local/lib,
set GEOS_DIR to /usr/local), or edit the setup.py script
manually and set the variable GEOS_dir (right after the line
that says "set GEOS_dir manually here".

I would recommend using anaconda to use basemap under windows using conda-forge. That should settle a lot of problems (basemap does not work with the most recent matplotlib version, see issue 495, installing a slightly older one should work, though).

If you want to work with a direct installation of Python without Anaconda under Windows, you may also use the wheel files generated by C. Gohlke available in https://www.lfd.uci.edu/~gohlke/pythonlibs/#basemap since they already provide the GEOS DLL for Windows. You can install them with pip by passing the direct path to the downloaded wheel file.
In the mid term I would like to upload such kind of wheels to PyPI, but I am currently doing a bit of refactoring first.

Finally (after one year!) we have precompiled binary wheels for Windows and GNU/Linux which can be installed with pip:
python -m pip install basemap

I am closing this issue now. Feel free to open a new issue if there are still problems with getting basemap work on Windows.


Look into pulling chat templates from huggingface

From discussion on #1002 from @bbrowning

Should we be using the chat templates as specified in the tokenizer_config.json of each model from HuggingFace instead of hardcoding them? Essentially https://huggingface.co/docs/transformers/main/en/chat_templating#how-do-i-use-chat-templates
This would get us out of having to hardcode the templates at all and instead we'll just use what the model's metadata says it requires. This would mean expecting models to maintain appropriate chat templates, but I believe that's a pretty common practice.

It looks like llama_cpp_python may automatically do this for us if we just don't pass a chat template in? At least https://github.com/abetlen/llama-cpp-python/tree/v0.2.55?tab=readme-ov-file#chat-completion seems to imply some ability here, but we might need to massage the settings we pass in to tell it to pull the metadata out of the gguf and/or huggingface tokenizer config file.

Noting that this is still relevant, as we are still hardcoding chat templates per model family at https://github.com/instructlab/instructlab/blob/92dae8665c82ef2611a7b6c92853fe2c425e5aaf/src/instructlab/model/backends/backends.py#L42-L51

I would like to work on this issue.

@Spartan-71 we welcome a PR! i will assign it to you :+1:

@Spartan-71 - what's the status on this?

closing as this has been addressed as part of https://github.com/instructlab/instructlab/pull/2592


Fix 403 Forbidden by providing most common user agents

Fixes #89

Fixes #89

I sent an email to @karan to see if he's willing to come take a look and merge PRs.
If we don't get an answer, maybe it's time to fork then? Who would be willing to host the fork?

My mail was not delivered, so I think we can move on and fork it.

Could you guys please indicate where you forked? I'd be happy to contribute too.

Forked into https://github.com/pawamoy/TPB. I still did not release to PyPI. I'll try to do it soon. I'd like to add other maintainers on the future PyPI namespace as well.

Released under the tpb namespace on PyPI: https://pypi.org/project/tpb/


Fix for #721

cleaner code
fixes #721

P.S.: @mholt sorry :cry:

@DenBeke You wanna give this a try? You could even just make the change locally on your computer if that's easier :stuck_out_tongue:
Thanks @elcore!


fix(ci): adjust CI so we don't accidentally overwrite and use a pinned base

the base images used to be build in this repository during releases. Looks like we never removed the logic to push the base image back to the repository when we moved the base images.
Also added a pin. Will try and automate the PR creation for bumping that pin separately

:tada: This PR is included in version 4.26.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Local auth provider implementation

@cjellick Includes implementation for local auth IdentityProvider based on plaintext comparison
Also this uses the types/Token CR changes PR https://github.com/rancher/types/pull/64
So vendor.conf points to my types repo.

merged in favor of https://github.com/rancher/auth/pull/6, which is the same with fixed up vendor


How to index a tarball to Elasticsearch

Output of kube-burner version
Version: 0.14.3
Git Commit: e8c3b246ac1c9dd52936f55e9b0b76d35d6eca32
Build Date: 2021-12-13T19:07:06Z
Go Version: go1.16.11
OS/Arch: linux amd64

Describe the bug
A clear and concise description of what the bug is.
Cannot seem to index a tarball to OpenShift Elasticsearch.
To Reproduce
Steps to reproduce the behavior:
kube-burner init -c api-intensive.yml  -u ${prom_url} -t ${token} -m ../../metrics-profiles/metrics.yaml
then:
kube-burner index -c api-intensive.yml  -u ${prom_url} -t ${token} -m ../../metrics-profiles/metrics.yaml
....
INFO[2022-10-01 10:54:39] Writing to: collected-metrics/clusterVersion-42f4b00a-8086-4101-8273-cda23a70e4de.json
INFO[2022-10-01 10:54:40] Metrics tarball generated at kube-burner-metrics-1643712879.tgz

kube-burner import -c api-intensive.yml --tarball kube-burner-metrics-1643712879.tgz
INFO[2022-02-01 11:09:58]  Creating indexer: elastic
FATA[2022-02-01 11:09:58] Unexpected ES status code: 500

I can access ES with a token :
sh-4.4# routeES=`oc get route elasticsearch -o jsonpath={.spec.host}`-nopenshift-logging
sh-4.4# token=$(oc whoami -t)
sh-4.4# curl -tlsv1.2 --insecure -H "Authorization: Bearer ${token}" https://${routeES}
{
"name" : "elasticsearch-cdm-XXXXX-1",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "XXX-ID-XXX",
"version" : {
"number" : "6.8.1",
"build_flavor" : "oss",
"build_type" : "zip",
"build_hash" : "Unknown",
"build_date" : "Unknown",
"build_snapshot" : true,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
    },
    "tagline" : "You Know, for Search"
  }

 cat api-intensive.yml
---
global:
  writeToFile: true
  createTarball: true
  metricsDirectory: collected-metrics
  indexerConfig:
    enabled: true
    esServers: [https://elasticsearch-openshift-logging.apps.domain.com]
    defaultIndex: kube-burner
    type: elastic
    insecureSkipVerify: true
....
(rest remains as the example in the Git)

Expected behavior
I can import the tarball and index it in Elasticsearch (maybe the config yml file includes a field for using a token in the authentication)

Hi @miiraheart, so it seems like the issue is related with the unsupported Bearer, at the moment only basic auth is supported for ES.

Hey @miiraheart , are you still interested in supporting Bearer authentication for ES indexing?

No activity in this issue for some time now... closing, make me know if you're still interested in that feature @miiraheart


Update SignalR 3.0 docs to use Endpoint Routing

Partially addresses https://github.com/aspnet/AspNetCore.Docs/issues/11127

Thanks for taking care of the merge conflicts for me.


chore: Replace taint node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane

…rnetes.io/control-plane

Reason for Change:

chore: Replace taint node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane
Issue Fixed:

Credit Where Due:
Does this change contain code from or inspired by another project?

[X] No
[ ] Yes

If "Yes," did you notify that project's maintainers and provide attribution?

[ ] No
[ ] Yes

Requirements:

[X] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[X] tested upgrade from previous version

Notes:

/azp run


nodestatus: Only list nodestatuses from the same namespace as the profile

What type of PR is this?
/kind bug

What this PR does / why we need it:
In case there were two profiles with identical names in two namespaces,
the operator would list them all, from both namespaces and then fail
with:
nodestatus.go:179 nodestatus "msg"="Not updating policy: not all statuses are ready"
"namespace"="default"
"nodeStatus"="sleep-sh-pod-ip-10-0-164-225.us-east-2.compute.internal"
"has"=18 "wants"=6
note that has > wants.
This patch adjusts the list so that only nodestatuses from the same
namespace are listed.
Which issue(s) this PR fixes:
None

Does this PR have test?
Yes

Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

NONE

Codecov Report

Merging #893 (a4a8fd7) into main (de3017a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #893   +/-   ##
=======================================
  Coverage   50.99%   50.99%
=======================================
  Files          42       42
  Lines        4536     4536
=======================================
  Hits         2313     2313
  Misses       2142     2142
  Partials       81       81

oops, looks like I broke something :-/
/hold

/test pull-security-profiles-operator-test-e2e

/hold cancel
this should be ready for another round...


updated 8-bit realm tileset

this is not done yet but it's done enough to at least start sharing it as i finish the design!
here are a few current sample states:

the way i've structured this tile is by moving 8-bit from "sets" to "quests"; i think the new 8-bit realm is complicated enough to 100% warrant inclusion as a quest, and i thought it would be good to make a "new" quest tile with tons of comments so that it is hopefully easier for me and current non-ezan non-fred devs to understand/unpack everything going on with quest tiles and all of ezan's little structural quirks around them.
right now, the starter tile is pretty simple. currently it operates as follows:

start with specific details about the quest. IE, "Gain 10000 points to grab your digital key!" in the front matter
similar to cyrpt tile setup, do a "BONUS ZONE: Vanya's Castle" type subentry that shows you the modifier to max, the points-per-fight, and advises the user of how far over or under the point max they are.
subentry near the end of the tile noting the # of turns left to get to 10000 assuming you max out all bonuses in all remaining zones

things i am on the fence about and would love thoughts on:

add an "optional tasks" tile that gives advice on maximizing the active bonus in the current 8bit zone. IE, the task generator would generate one for each zone but only publish the current bonus zone tile to tourguide proper. it would then give suggestions on increasing init/DA/meat/items for that zone.
if the user is not capping the bonus in their zone, add a subsection for the zone they'd be generating the most points in, and add text saying that if the user can't cap the bonus, they should adventure there for X points per turn.
out-of-run and after getting digital key, still generate this guy as an "optional" task so long as the user has < 30000 points; this way, users in aftercore wanting to get a pixel rock would be able to do so.

one other very small note -- i've deleted most of the now-useless 8bit stuff, but there's a resource generator function that might be good for low-shiny folks as it includes red pixel potions (which are still good for low-shiny shadow!) and there's a nag tile re: grabbing white pixels that actually still is useful in one specific path (kingdom of exploathing). i'm on the fence on whether i make a sets/8bit-realm.ash file and just plop that stuff in there, or whether i keep it in quests/8bit-realm.ash to keep the stuff in one file, even though quests very rarely generate ResourceGeneration blocks. thoughts?

oh here's one shitty thing -- this PR is using turnsSpent for turns-left-of-bonus. unfortunately, turnsSpent naturally advances on freeruns. so if the bonus doesn't advance on freeruns, that calc will be off in an annoying way for turns remaining. and i don't think it does. but i really don't think there's any way for us to fix this without a mafia pref specifically for 8bit zone killcount.

Tile and code look great, it'll be awesome to have this in soon.
I saw some of the comments in Discord about some things to do to slim down some of the vertical space and would echo those (combine current score with the top line, combine how many turns left in that zone into the BONUS ZONE line).
Re: your other questions:

"optional tasks" tile that gives advice on maximizing the active bonus in the current 8bit zone

I personally tend to just use the mafia maximizer. Are there any subtle things we'd be able to cover that wouldn't get noticed there?

If the user is not capping the bonus in their zone, add a subsection for the zone they'd be generating the most points in

Hmm. This might be confusing for people who don't realize that they may want to wait and tackle it later when they could cap the bonus in their zone. But if we use the right language it's probably fine.

out-of-run and after getting digital key, still generate this guy as an "optional" task so long as the user has < 30000 points; this way, users in aftercore wanting to get a pixel rock would be able to do so.

🤷 Up to you, I don't feel like that adds much value but that's my bias toward standard. It certainly won't hurt anything to add!
IMO any of those 3 would be okay to add but I would lean toward moving forward with this and adding them as enhancements later rather than holding up getting in the main tile for them.

there's a resource generator function that might be good for low-shiny folks as it includes red pixel potions (which are still good for low-shiny shadow!)

Yeah I used these for a long time, especially since funk-slinging isn't something you tend to perm super early. It'd be good to keep this. I don't have any strong opinions on exactly how we should organize it, up to your judgement.

re: your one main question...

I personally tend to just use the mafia maximizer. Are there any subtle things we'd be able to cover that wouldn't get noticed there?

a few. things like generating a vintner wine come to mind; no way maximizer tells you that. other things like using a mayday package for +init, crafting a lynyrd shirt, etc could come into play. but i generally agree that it's more a nice to have than anything strictly needed, and it would add a lot of cruft to this (as well as be a somewhat annoying thing to have to update all the time). totally fine ignoring that idea.
otherwise, i think the "persist the tile in aftercore for optional tasks" thing is fun but would require too much work on the tile for me to want to do it right now. certainly something i might do later but not a priority right now.
i'll refactor the resource stuff and then plop it in as one of our few quest-generating-resource-tiles lol


Adding a lazy-loading Shared buffer to both the Managed and Native pools

Adding a lazy-loading Shared buffer to both the Managed and Native buffer pools to reduce the boilerplate needed to use the pools if desired.
/cc @KrzysztofCwalina
#356

Though, this is not how you should implement Dispose. It's a bit convoluted, but here is the guidance: https://msdn.microsoft.com/en-us/library/b1yfkh5e(v=vs.110).aspx

@KrzysztofCwalina personally, I find that pattern terrible and thread-unsafe.
I've started using Interlocked.Exchange for these operations with a member variable tracking state. Usually something like
public void Dispose()
{
    IntPtr localHandle = IntPtr.Zero;
    if ((localHandle = Interlocked.Exchange(ref _nativeBuffer, IntPtr.Zero)) != IntPtr.Zero)
    {
        Marshal.FreeHGlobal(localHandle);
    }
}

Additionally, there's no guarantee that Dispose() will ever be called because it is doubtful this construct will be placed into a using statement. Therefore, the finalizer should call Dispose like
~NativeBufferBucket()
{
    Dispose();
}

@whoisj, making Dispose thread safe might make sense. In general, .net APIs are not guaranteed to be thread safe (with the exception of statics), but it's ok to opt into thread safety.
As to the finalized calling Dispose, it is not what we want APIs to do. There are many reasons for it, the main being that you should not be accessing managed objects from finalizers as they might be already finalized. Finalizer code path is very tricky to implement and should be super simple, ideally just a pinvoke call to release resource/handle.

looks good.

Thanks! Once the CI passes I'll merge

Thanks folks!


Adapt NetworkPolicys for components in garden namespace

How to categorize this PR?

/area dev-productivity ops-productivity networking security
/kind enhancement
What this PR does / why we need it:
The new NetworkPolicy controller is now enabled for the garden namespace. It is not necessary to put this behind a feature gate because all components in the garden namespace are controlled by Gardener directly.
Which issue(s) this PR fixes:
Part of #7352
Special notes for your reviewer:
/cc @timuthy @ScheererJ @oliver-goetz
Release note:

NONE

The upgrade tests will always fail in this PR due to the changed labels in the ETCD templates (https://github.com/gardener/gardener/pull/7636/commits/545bbcdfb6079237b3f5d57878b3e56a9603693d):
 - The StatefulSet "etcd" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden

The upgrade tests will always fail in this PR due to the changed labels in the ETCD templates (545bbcd):
 - The StatefulSet "etcd" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden

After this PR is merged, we can pin the "PREVIOUS_VERSION" to the respective commit sha in master branch like in https://github.com/gardener/gardener/pull/7245/commits/78e53b6aed3bd4fa78a3e83263e6175d3ab442e0#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52R36, and remove this again once v1.67 got released (https://github.com/gardener/gardener/pull/7398/commits/78e3ddae50f22a9745ef15097f176164363fefb3).

The ManagedSeed e2e test (part of pull-gardener-e2e-kind) fails because now that the gardener-resource-manager's NetworkPolicy controller is enabled for the garden namespace, our technical debt in the Seed deletion code pops up again: We don't explicitly delete resources deployed by the legacy Helm charts, i.e. the observability-related components. Hence, the controller is unable to remove its finalizer (resources.gardener.cloud/networkpolicy-controller) from the related Services which block the deletion of the garden namespace in the ManagedSeed.
In addition, the Service related to gardenlet itself also still contains this finalizer, however gardenlet will always be deleted only after gardener-resource-manager.
As a mitigation, I propose to

enhance gardenlet's Seed controller to delete its own Service before deleting the gardener-resource-manager.
enhance gardenlet's ManagedSeed controller to remove these well-known finalizers until the legacy Helm charts have been refactored to be deployed via ManagedResources so that all this can be deleted regularly.

/test pull-gardener-e2e-kind-migration


notebook | error linking extension

Hello! Is this project still being actively maintained? I'm running into the following error:
[W 2023-10-29 09:10:34.462 JupyterNotebookApp] 'contents_manager_class' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-10-29 09:10:34.863 ServerApp] notebook | error linking extension: The 'contents_manager_class' trait of <jupyter_server.serverapp.ServerApp object at 0x7f4ee1fff1c0> instance must be a type, but 'ipymd.IPymdContentsManager' could not be imported
    Traceback (most recent call last):
      File "/home/user/.local/lib/python3.10/site-packages/ipymd/core/contents_manager.py", line 28, in <module>
        from notebook.services.contents.filemanager import FileContentsManager
    ModuleNotFoundError: No module named 'notebook.services'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/traitlets.py", line 2152, in validate
        value = self._resolve_string(value)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/traitlets.py", line 2028, in _resolve_string
        return import_item(string)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/utils/importstring.py", line 31, in import_item
        module = __import__(package, fromlist=[obj])
      File "/home/user/.local/lib/python3.10/site-packages/ipymd/__init__.py", line 3, in <module>
        from .core.contents_manager import IPymdContentsManager
      File "/home/user/.local/lib/python3.10/site-packages/ipymd/core/contents_manager.py", line 30, in <module>
        from IPython.html.services.contents.filemanager import FileContentsManager
    ModuleNotFoundError: No module named 'IPython.html'

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/home/user/.local/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 342, in link_extension
        extension.link_all_points(self.serverapp)
      File "/home/user/.local/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 224, in link_all_points
        self.link_point(point_name, serverapp)
      File "/home/user/.local/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 214, in link_point
        point.link(serverapp)
      File "/home/user/.local/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 136, in link
        linker(serverapp)
      File "/home/user/.local/lib/python3.10/site-packages/jupyter_server/extension/application.py", line 405, in _link_jupyter_server_extension
        self.serverapp.update_config(self.config)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/config/configurable.py", line 244, in update_config
        self._load_config(config)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/config/configurable.py", line 193, in _load_config
        setattr(self, name, deepcopy(config_value))
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/traitlets.py", line 750, in __set__
        self.set(obj, value)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/traitlets.py", line 723, in set
        new_value = self._validate(obj, value)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/traitlets.py", line 756, in _validate
        value = self.validate(obj, value)
      File "/home/user/.local/lib/python3.10/site-packages/traitlets/traitlets.py", line 2154, in validate
        raise TraitError(
    traitlets.traitlets.TraitError: The 'contents_manager_class' trait of <jupyter_server.serverapp.ServerApp object at 0x7f4ee1fff1c0> instance must be a type, but 'ipymd.IPymdContentsManager' could not be imported

I'm glad to provide more information, in case there's interest in fixing it, though doing so on my own is beyond my Jupyter / Python knowledge.
Thanks!

My mistake. According to https://pypi.org/project/notebook/, "The Jupyter Notebook project is currently undertaking a transition to a more modern code base built from the ground-up using JupyterLab components and extensions."
This applies to version 7 onwards. Forcing the installation of version 6 solved the problem.
I was aware ipymd does not work with JupyterLab, but not that Jupyter Notebook was using JupyterLab. Apologies


iOS sample improvements

Changes primarily target the following issues:

v100.1 changed the mapView behavior resulting in additional invocations of the ViewDidLayoutSubviews method on the UIViewController. This caused some samples to become unviewable
Some samples did not appropriately adjust their layout upon rotation.
Some samples that used a toolbar to hold controls did not appropriately dismiss that toolbar upon leaving the sample.

@dkemlage Can you take a look at this when you get a free moment?

@nCastle1 The code looks good. Please Merge.