Implement 'public' access into OpenStack VPP test VMs

Feature: add support for public internet and ssh to vms
Tasks:

[ ] Add ability to publicly ssh into test VMs in VPP based OpenStack environment on Packet.net
[ ] Ensure public traffic does not share forwarding over the test network interface if possible

Acceptance criteria:
Pings and SSH between compute nodes works as expected

[ ] between VMs on the same compute node
[ ] between VMs on the different compute nodes
[ ] from the control host to any of the VMs
[ ] from the internet at large to any of the VMs
[ ] from any of the VMs to the internet

@robertstarmer please clarify if the VM has any network connectivity after setting up a public interface for internet access. It sounds like there may be no connectivity at all.
@iawells do you have any input on having multiple interfaces on a VM for OpenStack with VPP where the first is for public internet access and the others are for internal private traffic (with a VLAN for our test case)

@iawells You should have access to the control node of the cluster, where the router VPP config should be installed but doesn't look like it is.  vpp/vpp-agent is running, not throwing errors, and etcd and neutron-server appear to be happy as well.

@robertstarmer: First step is documenting how to make this work

A new ticket will be created for automated the Packet network side

This can be enabled with the “create_masq.sh” script and a “Public IPv4” network range being added to the control node.
Manual steps:

Go to the packet API/UI and request a /29 or larger subnet, and associate the entire range to the control node in the OpenStack cluster
Use the information from the range (‘0’ address for network, ‘1’ address to be associated with the controller host, and the rest becomes available to OpenStack for the ‘external’ router)
Use the create_masq.sh script with the information from the controller node to create the “netext” network with the Packet public IP Addresses defined.

Add documentation to https://github.com/cncf/cnf-testbed/blob/master/docs/openstack_deploy.md
Include information covering tasks and A/C in description

https://github.com/cncf/cnf-testbed/tree/feature-doc-updates


ModuleNotFoundError: No module named 'dpipe'

How to install dpipe? I tried to pip install dpipe but failed. I'm really interested in lora on conv. Could you please show the requirements?

Hi @supy0001! Thank you for pointing out, I have put it in now the main repository. You can install dpipe using the following:
git clone https://github.com/neuro-ml/deep_pipe.git
cd deep_pipe
git checkout 5c08d5759d51c0731cc636c2866bb3a538ffab7a
pip install -e .


Added OpenMP flags to FV3 build

I added option to enable OpenMP flag for @FVdycoreCubed_GridComp and @fvdycore in their respective CMakeLists.txt and propagated the newly created branch dev/aoloso/FV3_OpenMP_build all the way to GEOSgcm level modifying<EMAIL_ADDRESS>./src/Components/<EMAIL_ADDRESS>and Develop.cfg.
The intent for usage is to add -DOPENMP_FV3=YES (or 1 or ON or TRUE or Y) to the make step when enabling OpenMP is desired for FV3 dyn core.

@aoloso I think this was good for getting the CI to be successful, but I don't think it can ever be accepted. The real PR would have to be to the CMakeLists.txt on the GEOS development branches of the FVdycoreCubed_GridComp and GFDL_atmos_cubed_sphere repositories. Once those are accepted, then the develop branch of GEOSgcm_GridComp would automatically get the option. Changes to Externals.cfg and Develop.cfg are only really allowed in structural and real-make-a-new-release by @sdrabenh changes.
And speaking on that, I'm not sure the option was handled how we'd want. What we'd want is to add a CMake option() ability a la:
option (OPENMP_FV3 "Enable OpenMP compilation in FV3" OFF)

But, that said, I/we might need @tclune and @pchakraborty to think on this as to where is the best place for this option() and the best way to handle it. Since FV3 is used by multiple fixtures, and sometimes without, say, GEOSgcm_GridComp we'd need a basic enough CMakeLists.txt (or .cmake file) to allow. Perhaps ESMA_cmake/esma.cmake? I guess nearly any GEOS fixture that needs FV3 would also have it and then it would gain the ability.


Favourite icon should update if item removed from 'Favourites' list

The favourite icon doesn’t alter on the photo page when I’ve removed a photo from my ‘Favourites’ list, even after a reload. It is kind of hard to tell with that icon, though, maybe reconsider the colour choices?

That's expected, given how I implemented it. It should fix itself when I modify #101, but I'll leave this open to ensure it does the right then when complete.


Add that extension also accepts a .graphqlconfig file

Currently the README marks that the project NEEDS a .graphqlconfig.yml file when in fact it also supports a .graphqlconfig. Confirmed it locally to also support it.

@intellix : Thanks for the PR 🙌


Added settings popover, conditional redirection for instructor evals

Yay! First PR! First contribution!! Great work!


Set the endian of a bytebuffer

As for now the bytebuffer only reads and writes in big endian order. In some cases one might actually work on the data using little endian (I'm currently facing this problem with a old archive file type). So I would consider adding a way to change the buffers order for example using a set_endian method.

Good point, I'll add that option.
Or you can submit a PR if you want to :)

Alright I gonna work on that and submit the PR the next days. Thanks for your fast response.

Np, I'm glad this lib is useful to someone :)
Don't hesitate to ask if you need anything !


Update manifests/t/Tencent/TencentVideo/11.72.3224.0/Tencent.TencentVideo.locale.en-US.yaml

Fixes an issue where this package has multiple different monikers
Microsoft Reviewers: Open in CodeFlow

[Policy] Reset Labels


Problematic Dependencies in cloud_admin

When running cloud_admin on the CERN machine I used to get the error No module named httplib2. I've installed httplib2 and various other modules that cloud_admin asked for. Finally, I encountered this impasse:

I want to install the module discovery
The module discovery requires requests-2.5.0
With requests-2.5.0 the cloud_admin and cloud_status commands fail because keystoneauth1 requires requests>=2.10.0
I upgrade requests
Everything works but cloud_admin is asking for the module discovery

Can we remove that dependency?

does it requires that exact version?
if not you could try doing:
"pip install  -force-reinstall -upgrade"
and to try and get pip to install the latest versions of things instead of using the currently installed version

The discovery module is related to googlecompute. since there seems to be some weird version requirements and we're not using gce right now one workaround would be to just comment out the googlecluster imports


53 value of the shrared secret resetting to its default value shared secret 1 when you exit from science mesh setting page

Last commits:


1.10 -> master branch merge

Is there any chance of a branch merge please?

I was already doing it, just needed to change some fixers to be compatible with 2.x.
Done.

Awesome. Thanks a lot. :)


Publish option ExcludeObjectTypes not working with multiple values

hello
i have issue when setting option to drop objects not in source and trying to preserve security objects for instance Users and Views seems like after semi-column not take value, tried with quotes and double quotes without success. When setting one object type works!
 dotnet publish /p:TargetDatabaseName=API_clean /p:DropObjectsNotInSource=True /p:ExcludeObjectTypes=Users;Views /p:BlockOnPossibleDataLoss=False
...
Setting property DropObjectsNotInSource to value True
  Setting property ExcludeObjectTypes to value Users
...
 Dropping Permission...
  Dropping [dbo].[aaa]...
  Dropping [dbo].[V_aaaCustomer]...
...

With sqlpackage.exe it does work but use different sets of flags
    <DoNotDropRoutes>True</DoNotDropRoutes>
    <DoNotDropRules>True</DoNotDropRules>
    <DoNotDropUsers>True</DoNotDropUsers>
    <DoNotDropSignatures>True</DoNotDropSignatures>

thanks in advance for your help

@pada57 Just a quick question, have you tried to put the ExcludeObjectTypes property in the project file and do you get the same problem there?

@pada57 Just a quick question, have you tried to put the ExcludeObjectTypes property in the project file and do you get the same problem there?

yes same using ExcludeObjectTypes or DoNotDropObjectTypes on project file XML
i will use sqldeploy.exe anyway as not part of this project

I did some digging into this issue and it looks like semicolon separators for these values are a bit problematic due to how they are parsed by shells. I have an implementation that works with comma's instead and that works just fine.
Technically that would be a breaking change though, but since this property (and others as well) can't be used currently I'd rather not make the fix for this a 2.0. @ErikEJ @jeffrosenberg What do you think?

I did some digging into this issue and it looks like semicolon separators for these values are a bit problematic due to how they are parsed by shells. I have an implementation that works with comma's instead and that works just fine.
Technically that would be a breaking change though, but since this property (and others as well) can't be used currently I'd rather not make the fix for this a 2.0. @ErikEJ @jeffrosenberg What do you think?

Is it not just a bug fix then?

Is it not just a bug fix then?

Yeah, except that if anyone had DatabaseSpecification, DoNotDropObjectTypes or ExcludeObjectTypes  in their project files using semi-colons they would now have to change that to be comma separated or it won't work anymore. But then again, it doesn't work currently so I guess we can treat this as a bug fix anyway. I'll send in a PR soon.

Yeah, except that if anyone had DatabaseSpecification, DoNotDropObjectTypes or ExcludeObjectTypes  in their project files using semi-colons they would now have to change that to be comma separated or it won't work anymore. But then again, it doesn't work currently so I guess we can treat this as a bug fix anyway. I'll send in a PR soon.

Add a check for the presence of semicolons and throw?

Add a check for the presence of semicolons and throw?

@ErikEJ That's not a bad idea actually ;). I'll add that in.

@ErikEJ That's not a bad idea actually ;). I'll add that in.


[Bug] Bookie instanceId verification failed

Search before asking

[X] I searched in the issues and found nothing similar.

Read release policy

[X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version
image: 3.0.4
k8s deploy
Minimal reproduce step
First, the cluster bookies prompts that it cannot get instance_id and asks me to use bin-bookkeeper shell metaformat to initialize. Then restarting bookies reports this error. I take the instanceId 237fe99b-9a46-4697-bff1-4739d68a9175 is not matching with 52088892-8876-4c1c-b6fa-cf00e487d985 prompt 237fe99b-9a46-4697-bff1-4739d68a9175 to the bookies host to modify the VERSION file. Then restarting it still has the same error, but this time it no longer prompts the instanceId to be modified.
What did you expect to see?
I expect it to start normally
What did you see instead?
org.apache.bookkeeper.bookie.BookieException$InvalidCookieException:
Anything else?
Solution
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

image: 3.0.4
k8s deploy

@HGHNice Which method is used for the k8s deployment? Is it by Apache Pulsar Helm Chart? Which version of the Helm chart?

image: 3.0.4
k8s deploy

@HGHNice Which method is used for the k8s deployment? Is it by Apache Pulsar Helm Chart? Which version of the Helm chart?
I'm glad you replied to me. This picture shows the version of helm-chart I use. I would also like to ask how to scale down bookies if I need to.


Sitemap Caching – sitemap.xml doesn’t update on Netlify host unless cacheMaxAgeSeconds is set to 0

Hello!
So this is a fairly simple one to describe, I’m not sure if I’m missing something?
My first instinct was to blame Netlify, but I realised if it is working perfectly with cacheMaxAgeSeconds: 0 then it must be something more complicated at play.
"@nuxtjs/sitemap": "^5.2.0",
"nuxt": "3.12.1",
Here is my nuxt.config.ts setup:
  sitemap: {
    cacheMaxAgeSeconds: 0,
    sources: [
      '/api/sitemap'
    ]
  },

and here is the referenced api/sitemap function – this works perfectly as an endpoint; it updates instantly even on Netlify.
import { createClient } from '@sanity/client'
import { resolveHref } from '../../utils/resolveHref'

const client = createClient({
projectId: process.env.SANITY_PROJECT_ID,
dataset: process.env.SANITY_DATASET,
useCdn: false,
});

export default defineEventHandler(async () => {

const query = `*[_type in ['page', 'exhibition', 'event']] {
  "slug": slug.current,
  _type,
  _updatedAt,
}`;

const documents = await client.fetch(query);

const urls = documents.map(doc => ({
  url: resolveHref(doc._type, doc.slug),
  // url: doc.slug,
  lastmod: doc._updatedAt,
  changefreq: 'weekly',
  priority: 0.8
})) || [];

return urls;
})

Here is the hosted sitemap:
https://staging--tinworks.netlify.app/sitemap.xml
Thank you for any guidance :-)

Hi, thanks for the details.
This sounds like the default behavior which is to cache the sitemap for 10 minutes (see Sitemap Caching).
Since the sitemap is hit quite often by robots we can reduce the cost of running our site by caching this data. You can usually see the cache status by inspecting the response headers, the URL you provided looks like caching is off though
If this is caching longer than 10 minutes though it sounds like a bug.

Hey Harlan!
Thanks so much for your work on this and sorry for missing that detail – I had read the docs and understood that there was a cache by default.
I checked with a variety of times including unset, 10 minutes set explicitly and then 1 second / 10 seconds to be sure it wasn’t working. In the case of the 10 minute wait I did wait 20 minutes incase I was misunderstanding when the cache was being reset.
Would it be helpful if I set a cache time again and inspected the response headers?
I did also visit the cached Sitemap in private session / different browser incase there might be any issues with my browser config.
S

The only catch may be that the cache is SWR so it will serve old sitemaps after 10 minutes and refresh in the background. Otherwise, it should be resetting.
Any reproduction would be useful to debug further.

So if this is still an issue the only thing I think I can do on my end is set a cache time to 0 seconds by default for Netlify and add a warning if users set a cache time.
Let me know your thoughts, it's a bit strange no one else has reported this though.


PR: "ACES" Computer Graphics Config Generator

Hi,
Here is a PR for the foundations of ACES Computer Graphics (CG) Config Generator. As for the aces-dev Reference Config Generator, the generation
is driven by a mapping CSV file although the process is different:

The CG config inherits from the Reference config data.
The mapping CSV file excludes unlisted transforms by using their ACEStransformID.
The mapping CSV file requires builtin CLF transform by using their CLFtransformID.

There is a lot of new code to:

Serialize/Deserialize the config Python data to JSON.
Generate CLF transforms.

There are a few dummy/non-dummy ones included for the CG config, the generation relies on Colour / Numpy.

Classify CLF transforms in a similar way to the CTL transforms.

No graph code, it is not really required but might be useful.

I think that at this point, the core functionality should be working, I haven't
tested the config and all the CLF transforms need to be implemented. The CG config code should be reusable for the Studio config and will have to move somewhere else.
I have autocratically decided on various aspects of the CLFtransformID, trying to stay close to aces-dev but it is certainly worth discussing... naming convention, who does not enjoy that? :)

CG Config: https://gist.github.com/KelSolaar/a52ddcc0e546749bb6671af8e0582838/0c9f85fc39e782713477c91d92b0df03fe2309ce
CLF files in the commits.

Voila!
Thomas

There are a few things we might want to add either here or in a subsequent PR (might be better to get that one merged):

Support for NamedTransform (I was planning to add it).
Support for BuiltinTransform via the CG spreadsheet according to the last meeting.


Ensure repos don’t restore test-only packages during source-build

Confirmed that after https://github.com/dotnet/source-build/pull/847, no test-only prebuilts remain in the report. A couple of files are still caught by test project heuristic, tracked by https://github.com/dotnet/source-build/issues/846.


[Tutorial proposal] INTRODUÇÃO AO GOOGLE EARTH ENGINE

What is the objective of the proposed tutorial?
In a first step, understand the Code Editor and explain the basics of JavaScript to use in GEE. Then, apply the API features and lastly, build an application..
What is the scope of the proposed tutorial?
Forest, water resources and specific geographic - Iguassu Falls - Paraná, Brazil..
Please provide an outline of the structure of the proposed tutorial?

Interface
a. Code Editor
Introduction to JavaScript
a. Data Types in JavaScript
b. Constraints on naming variables
c. Accessing and Changing an Array
d. Expressions and Operators
e. Conditionals, Condition Operators and Logical Operators
f. Switch Structure
g. Loops
h. Functions
i. Objects
Earth Engine Objects
a. Strings
b. Numbers
c. Lists
d. Casting
e. Dictionary
f. Dates
Hands-on: Viewing Images and Image Bands
a. View Customization
b. Adding Colors
Hands-on: Calculations
a. Map Algebra
b. Image Statistics
Image Collections
a. Filtering
b. Composition with reducers
App with GEE

In what format will you be submitting the tutorial?
Markdown
This request will be reviewed by the Earth Engine community maintainers, who
will reply on this issue tracker with any questions or suggestions. Once
approved, this issue will be assigned to you and you can begin work on the
tutorial following instructions in
Writing a tutorial.
When creating your Pull Request, enter "Closes #issueno" in the description of
your Pull Request to link the tutorial to this issue.

Hello how are you?
The idea would be in Portuguese - BR, because here in Brazil this technology is still little used and needs greater dissemination.
Many people from academia and the market would like to learn.

Hi @wesleymarchi, so sorry for the delay.  We'd love to receive and review your tutorial. You can find instructions here on how to get started. Let us know if you have any questions!

@wesleymarchi Hey, man! how are you?
I'd love to help you with this.
If you need anything, I'll be happy to help

Good afternoon, how are you?
Sorry for the delay in responding, there was a lot of demand for postgraduate studies. I will start this demand.
Thank you!

Hello, is there anything we can do to help move the process along? If it is something that you don't think you'll be able to get to, it is okay to close the issue as well.


Duplicate IP addresses at scale: Possible read/write locking problem?

@pperiyasamy has reported that they're seeing duplicate IP addresses on a large k8s cluster, using the k8s backend.
He's pointed out that a lock may not be acquired like it is in the etcd implementation (the same problem does not appear when using the etcd backend)
https://github.com/k8snetworkplumbingwg/whereabouts/blob/master/pkg/storage/storage.go#L82-L108
Does this need to have a further locking mechanism?
How to test for the duplicate IP problem with this scaled up concurrency?

/cc @JanScheurich

The Kubernetes CRD backend does not use locks, but does use what was at the time a recommended alternative: https://github.com/k8snetworkplumbingwg/whereabouts/blob/5e8cafde59fc910f032df1e58997751920957ca4/pkg/storage/kubernetes.go#L182-L199 -- if I can find a reference I'll add a few links. IMO, using annotations / resource versions and the jsonpatch "transaction" matched closely to how locks are often implemented with etcd.
The intent here was to use JSON patch for the update operation, with added tests to:

Only allow the update operation to occur on the expected resource version

this should prevent two operations from being written at the same time

ensure patched paths (adds) are empty

this seemed like an extra safeguard that was good to add

Perhaps something is going wrong with the patch, or how the tests were assumed to work.
How many IPs are in use when this occurs? Could we be hitting our upper bound? I know that we have one, but I have not looked into it before.
I am a few versions behind, and only use a few IPs so may not be hitting this edge case.

We see this issue massively (~5 duplicates per IP address) already with a 100 pod deployment in a cluster of 80 node when using a huge /8 IPv4 range. So lack of available IPs we can rule out. In the kubelet logs we observed a huge number of where about retries due to failed patch operations. Apparently the concurrency protection of the patch is working at times but does not prevent duplicate assignments fully. I can't rule out that we hit the upper limit of retries, but If I look at the code that should lead to IP address assignment failure. We don't see that.

The intent here was to use JSON patch for the update operation, with added tests to:

Only allow the update operation to occur on the expected resource version

this should prevent two operations from being written at the same time (and retry logic should allow eventual success)

ensure patched paths (adds) are empty

this seemed like an extra safeguard that was good to add

Perhaps something is going wrong with the patch, or how the tests were assumed to work.

@crandles I hope this still makes write to succeed just by overwriting an existing ip address mapped to another pod in p.pool.Spec.Allocations. that's not wanted, isn't it ?
As Jan mentioned, I think we should use similar approach as implemented for etcd datastore type.

@pperiyasamy : I have not found any specific documentation about the patch operation of the K8s API that would describe the meaning of tests within a patch and how the K8s API reacts to failing tests. My naive interpretation of what should happen inside the k8s API server is:

Lock the K8s resource to be patched in etcd
Read the locked resource
Check the tests with respect to the locked resource
4a. If all tests succeed, apply the patch to the resource, write it to etcd and remove the lock. Return success
4b. If any of the tests fails, do not apply the patch, remove the lock and return failure.
With the specific test on the original resource version, this should only let one of a set of conflicting concurrent patches succeed (assuming, of course that the resource version is stepped with every update). The remaining patches should fail and trigger a complete retry (read, modify, patch).

Conceptually this is similar to a spin lock using atomic compare&swap instructions, except here all of the involved operations are heavy on the K8s API. So this doesn't seem like a good approach in scenarios where many transactions happen in parallel, as is systematically the case when K8s replicasets, statefulsets or daemonsets are deployed/deleted with many pod replicas.

re: upper limit, i meant limit on number of IPs we can allocate. This is going to depend on the maximum size of CRD resources. I'm not sure what the maximum number of address we can store in one pool is.

@crandles I hope this still makes write to succeed just by overwriting an existing ip address mapped to another pod in p.pool.Spec.Allocations. that's not wanted, isn't it ?

Correct, but locking doesn't prevent that either.
I just looked at the lock implementation in etcd, and I think we could do something similar by adding an owner or similar annotation to lock the resource, if it is present watch and wait, and releasing it upon completion.

Thanks for the collaborative exchange, and Chris I really like the idea about adding a locking mechanism with an "owner" annotation -- I think that's probably pretty feasible.
Tomo also mentioned the possibility of using some kind of leader election type of mechanism, he had linked:

https://pkg.go.dev/k8s.io/client-go/tools/leaderelection
https://github.com/codecentric/kubebuilder-starwars-example/blob/master/vendor/sigs.k8s.io/controller-runtime/pkg/leaderelection/leader_election.go

@dougbtv I too think so, the locking mechanism with an "owner" annotation looks to be simple approach as it can just follow Lock and Unlock syntax which would make Store APIs to be in intact. I guess there won't be still any race conditions with this approach.
leader election is also a good option, but it might break current Store API contract as the IPManagement code has to be executed in LeaderCallbacks as per this example

Just an FYI that this problem has been under active development.
Essentially, what it comes down to is that Peri and Tomo have discussed at length designs for moving forward, and we have proposed a number of fixes which have been locally tested for general usage, but are awaiting some tests at scale -- this is non-trivial to replicate locally.
Tomo has proposed a leader election pull request @ https://github.com/k8snetworkplumbingwg/whereabouts/pull/113
Peri has two pull requests available @ https://github.com/k8snetworkplumbingwg/whereabouts/pull/114 & https://github.com/k8snetworkplumbingwg/whereabouts/pull/115 which use a "lock" CRD to denote that a particular instance has acquired the lock via CRD.
Tomo's main consideration is that using the k8s libraries to solve for this issue ensures long-term maintainability. Peri has proposed a change to the way we use Update() although this seems to cause some blind load on the cluster. Peri has also proposed pull requests which use the lock CRD as an alternative and to assess performance facets.

During Peri's and my testing we found out that the actual root cause for the duplicate IPs was an incorrect handling of the K8s client context timeout. Instead of an error, the current IP was returned to the CNI at context timeout, despite the fact that the IP address was not successfully marked as allocated in the IP pool.
With a corrected context timeout handling, both the original Patch API and the simplified Update API were working correctly without causing duplicate IPs.
We then continued testing to compare the behavior and performance of the different proposed solutions:

Existing Patch API with correct context timeout handling (#119)
Update API with correct context timeout handling  (#114 )
IPPoolLock protecting the Update API (#115)
Tomo's Leader Election solution (#113)

As test case we create and delete a K8s deployment with 500 pod replicas on bare metal cluster of ~70 worker nodes. Each pod had a single secondary OVS CNI interface with whereabouts IPAM. Without any whereabouts patch this always resulted in a significant number of duplicate IPs.
A major consequence of corrected context timeout handling is that the CNI ADD or DEL commands now fail, triggering Kubelet to retry the pod sandbox creation or deletion. We observed that Kubelet apparently retries sandbox creation infinitely, but sandbox deletion only once.
For a deployment of 500 pods that can give rise to thousands of FailedCreatePodSandBox pod Warnings with reason "context deadline exceeded". Eventually the deployment succeeds, but it takes quite long and the event noise doesn't look good.
The times it took until all 500 pods are in Running state is:

Existing Patch API with correct context timeout handling:      5m 25s
Update API with correct context timeout handling:                5m 30s
IPPoolLock around the Update API:                                         4m 04s
Tomo's Leader Election solution:                                              35 mins

In contrast to 1-3, the leader Election solution took 7-8 times as long but produced only 34 "context deadline exceeded" Warning events and Kubelet retries.
Worse than the warning noise during pod creation is that solutions 1-3 all leave approx. 450 stale IPs in the pool when the deployment of 500 pods is deleted. The root cause are again the context timeouts together with fact that Kubelet retries deletion of the pod sandbox only once and then deletes the pod no matter if the CNI DEL was successful or not.
The Leader election patch did not leave stale IPs or "context deadline exceeded" Warnings but took in the order of 12 minutes to delete all 500 pods.
We believe the large number of context timeouts of solutions 1-3 are a consequence of the many quasi-parallel K8s API operations triggered by the independent but highly synchronized CNI ADD/DEL operations. As each CNI retries its failing K8s API operation many times, the API rate limiter kicks in and slows down the requests even more. We have tried with several back-off schemes in whereabout's K8s storage backend, but didn't manage to improve the bad behavior significantly.
All in all, the tests have confirmed our suspicion that the K8s API server with its optimistic locking and client retry paradigm is a particularly bad choice as storage backend for whereabouts IP pool use case. The etcd backend with its blocking lock API is much more efficient and suitable.
The leader election approach appears to be working more correctly and with less noise, but its performance still seems unacceptably slow. We haven't investigated that further, yet.
To rely on the K8s API server as storage backend, we'd probably need to redesign the whereabouts IPAM along the lines of Calico IPAM to assign IP blocks to individual nodes through the K8s API server and manage the IP blocks locally on each node without further use of the API server.

We have tried #127 on a 100 node setup with deployment/undeployment of 500 pods with 2500 ms backoff configuration. It solved the problem of both duplicate IP assignment and stale IPs. Following was the time taken -
a. Deployment - 4 min 34 seconds
b. Undeployment - 5 min 49 seconds

@dougbtv Has this issue been fixed in the latest whereabouts release 0.5 or 0.5.1?

Just a note that 0.5.0 and 0.5.1 should have the race condition fixes, I recommend 0.5.1 as it also has some bug fixes for ip reconciliation.
https://github.com/k8snetworkplumbingwg/whereabouts/releases/tag/v0.5.1

I have same ip conflict issue in my k8s environment using whereabouts v0.6.3.
Our k8s network use multus-cni + macvlan RoCE  solution, each pod may have 4 additional network interfaces...there are many ip conflict pods after about 1-2 weeks running.
To fix the ip conflict issue, I read the whereabouts codes, I found that  whereabouts  has several bad design :

use a large configmap maintaining ip assign records, which doesn’t work good in large network, especially in  16 cidr network.
use daemonset instance for leader election, and iterate for the whole ip range to allocate just one ip, which may lead to ip conflict in a high concurrent case.

So I wrote my own etcd backend ipam ,  so far works well in my k8s environment.

@yowenter, please try with version v0.8.0, we fixed some issues regarding ip conflicts


Questions on compute_down

I'm implementing an (internal) backend for Blaze, and I want to intercept all Selection, Projection, and Join expressions when computed against a new type of data.  compute_down seems to be the logical choice here, except that it's not triggered when I want when computeing an expression.
This works fine:
s = bz.symbol('s', 'var * {a: int32, b: string}')
e = s[s.a > 100]
bz.compute(e, my_custom_backend_type)

Because e is a Selection object, compute dispatches to my compute_down implementation, and all is well.
Adding a simple sort breaks things, however:
sorted_e = s[s.a > 100].sort('a')
bz.compute(sorted_e, my_custom_backend_type)

From tracing the code, to compute sorted_e, Blaze first checks for a compute_down implementation for Sort against the custom backend type in top_then_bottom_then_top_again_etc().  Since that isn't implemented, Blaze then calls bottom_up_until_type_break(...), which descends all the way to the leaf terms and computes them before computing their parents.  Only if there is a type break will Blaze re-compute from the top level again, providing an opportunity to dispatch the Selection compute_down implementation.  That never happens.
For this backend, I need to always compute Selection, Join, and Projection before their subterms.  What I want Blaze to do, is to try calling compute_down while walking the expression tree in bottom_up_until_type_break, and if it can sucessfully dispatch to compute_down, then return that result right away.  That would ensure that my compute_down implementations are always triggered before their subterms, which is what I need here.
Some questions:

is there another way to accomplish what I'm after here that doesn't involve compute_down?
my concern is that adding a call to compute_down inside bottom_up_until_type_break may break existing backends.  I can experiment with this to see.  Is there a fundamental flaw in this approach?

This is all consistent. we only compute down at the start or after a type change. You can use optimize to replace these nodes with a custom subclass that tracks the other type of data. This is very close to what we do with SimpleSelects

@llllllllll I accidentally hit "comment" before finishing my (long winded) post -- could you look at the rest of it and let me know your thoughts?
Thanks.

Ah, what I would do is have compute_up for that node return a box type such that it forces the compute_down implementation to get called, for example:
@dispatch(Selection, ndarray, ndarray)
def compute_up(expr, arr, pred, **kwargs):
    return Box(expr, arr, pred)

@dispatch(Selection, Box)
def compute_down(expr, box, **kwargs):
    ...

does this do what you want it to?

@llllllllll thanks, that makes sense, but unfortunately it doesn't fit our usecase.
Almost every instance of compute_down dispatches on the Expr base type and a specific data type.  This works well for type-specific dispatching, since blaze's compute logic will break whenever there's a type change, which re-triggers compute_down.
But it doesn't work well when we want to dispatch on specific Expr subclasses.  For these cases, compute_down has to  traverse the expression tree manually and make the node substitutions.  This feels broken to me, since we're re-implementing tree traversal in many places.
In our usecase, with Selection, we specifically do not want blaze to compute the predicate before we have a chance to analyze it.  I'd rather not traverse the whole expression tree in compute_down to find all Selections; I'd rather let the core compute logic handle tree traversal for me.
In your example, compute_up will hit Selection after the selection's predicate is already computed.  We specifically don't want this behavior.
We can get around this by dispatching on Expr, CustomType, CustomType and walk the expression tree ourselves to intercept the Selection nodes before their predicates are computed.
But it feels weird to have to reimplement tree-traversal logic in several places; it feels like, from a design point of view, Blaze's compute infrastructure should be pretty much the only place where tree traversal logic goes, unless there are very special cases to handle.
This also comes up in blaze.cached.CachedDataset: it ends up being pretty much useless.  It only works when computing the exact same expression against the cache; if computing a new expression based on a cached expression, the underlying cached dataset isn't used at all, leading to expensive re-computation.
This is another case where, if blaze called compute_down while walking down the expression tree, then it would be much more useful, since we would have an opportunity to insert the cached result as high up in the expression tree as possible.
I'll play with tweaking the compute_down logic in bottom_up_until_type_break to see how badly my proposed changes will break existing code.  That will help determine what I'm thinking will work or not.  If things break badly, then I'll probably try another route.

If you want the exact case of not computing the predicate before the table then this already exists in blaze.expr.optimize.simple_selections. An example usage of this is in the mongo backend. The only concerns I have with calling compute down at every step is performance, but that is not that big because I am mainly using blaze with postgres with non-trivial queries that dominate the runtime, but also that it blurs the distinction between compute_down and optimize.

@llllllllll Thanks for your explanations; I was able to implement what I needed without resorting to changing the core compute logic.

Awesome!


Cant run example of SepTop

I'm trying to run the example : https://psivant.github.io/femto/latest/guide-septop/
when:
complex_structure, complex_system = femto.fe.septop.setup_complex(
    config,
    receptor,
    ligand_1,
    ligand_2
)

error happen:
AttributeError: 'SepTopConfig' object has no attribute 'restraints'

the config is defined by:
import femto.fe.septop

config = femto.fe.septop.SepTopConfig()

use of   config.complex.setup instead of config
complex_structure, complex_system = femto.fe.septop.setup_complex(
    config.complex.setup,
    receptor,
    ligand_1,
    ligand_2
)

The above error no longer occurs.
but new error happen when: receptor_ref_idxs_2 = femto.fe.reference.select_receptor_idxs:
IndexError: index 53 is out of bounds for axis 1 with size 42

ligand_1, ligand_2 = femto.md.system.load_ligands(
    ligand_1_coords=eralpha_dir / "forcefield/2d/vacuum.mol2",
    ligand_1_params=eralpha_dir / "forcefield/2d/vacuum.parm7",
    ligand_2_coords=eralpha_dir / "forcefield/2e/vacuum.mol2",
    ligand_2_params=eralpha_dir / "forcefield/2e/vacuum.parm7",
)

When I replaced 2d and 2e with 3a and 3b, the error disappeared.
It's weird. I don't know why that is.

use of config.complex.setup instead of config

good catch!

but new error happen when: receptor_ref_idxs_2 = femto.fe.reference.select_receptor_idxs:

this should be fixed by #16!


Reduce marketplace refreshes significantly

Playing Today with up to ten players and the marketplace refresh time is too long and is disrupting the flow of the game.
Propose 30 minute refreshes with 10 minute bid times or something close this.

Note also speaking about changing the time limit to transaction count based not time based

I am marking this as obsolete since marketplace has changed to adventurer based (not shared)


Stream doesn't have on

I'm using gulp babel with sourcemaps pretty much as indicated by the README:
var sync       = require( 'browser-sync' );
var gulp       = require( 'gulp' );
var sourcemaps = require( 'gulp-sourcemaps' );
var babel      = require( 'gulp-babel' );
var concat     = require( 'gulp-concat' );

gulp.task( 'compile', function(){
	gulp.src( 'src/**/*.js' )
		.pipe( sourcemaps.init() )
		.pipe( concat( 'src/js/index.js' ) )
		.pipe( babel() )
		.pipe( sourcemaps.write( '.' ) )
		.pipe( gulp.dest( 'app' ) )
		.pipe( sync.reload );
} );

gulp.task( 'serve', function(){
	sync( { server: './app' } );

	gulp.watch( 'src/**/*.*', [ 'compile' ] );
} );

gulp.task( 'default', [ 'compile', 'serve' ] );

Everything is compiled and mapped as I desire, but the compile task ends violently:
$ gulp
[15:15:29] Using gulpfile ~/Documents/git/ithildin/gulpfile.js
[15:15:29] Starting 'compile'...
[15:15:29] 'compile' errored after 7.7 ms
[15:15:29] TypeError: Object function () {
        var args = Array.prototype.slice.call(arguments);
        if (singleton) {
            return singleton[name].apply(singleton, args);
        } else {
            if (name === "reload" && args[0] && args[0].stream) {
                return utils.noopStream();
            }
        }
    } has no method 'on'
    at DestroyableTransform.Stream.pipe (stream.js:65:8)
    at Gulp.gulp.task.sync.server (/Users/barney/Documents/git/ithildin/gulpfile.js:16:4)
    at module.exports (/Users/barney/Documents/git/ithildin/node_modules/gulp/node_modules/orchestrator/lib/runTask.js:34:7)
    at Gulp.Orchestrator._runTask (/Users/barney/Documents/git/ithildin/node_modules/gulp/node_modules/orchestrator/index.js:273:3)
    at Gulp.Orchestrator._runStep (/Users/barney/Documents/git/ithildin/node_modules/gulp/node_modules/orchestrator/index.js:214:10)
    at Gulp.Orchestrator.start (/Users/barney/Documents/git/ithildin/node_modules/gulp/node_modules/orchestrator/index.js:134:8)
    at /usr/local/lib/node_modules/gulp/bin/gulp.js:129:20
    at process._tickCallback (node.js:419:13)
    at Function.Module.runMain (module.js:499:11)
    at startup (node.js:119:16)

Any ideas?

The error is coming from your serve gulp task and from most likely the sync call which is completely unrelated to gulp-babel.

But the log is specifically telling me that the compile task errored. It doesn't even get to serve…

at Gulp.gulp.task.sync.server (/Users/barney/Documents/git/ithildin/gulpfile.js:16:4)

:O weeeird. Well-spotted, thanks. I'll post back when I've tracked this down.


external-dns provider for powerdns api

It would be nice if the external-dns service could support some commonly used internal dns servers like powerdns.

+1

This is a fine idea but not of any immediate use to us or paying customers (that I know of) so it would fall into the "PRs accepted" category.  We're happy to help if someone has questions on integrating an implementation into external-dns.

That could be a great idea thx ingardm

This seems to be implemented by @jgreat and merged in 3 month ago via PR#40 by @alena1108.
@vincent99 This issue can be closed, I think.


Added all ImplicitlyAnimatedWidget widgets support

Description
This PR adds support for all the inherited widgets from ImplicitlyAnimatedWidget. Also adds support for Directionality widget.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Code Style Guide] and followed the process outlined there for submitting PRs.
[x] All existing and new tests are passing.
[x] I updated/added relevant documentation (doc comments with ///).
[x] The analyzer (flutter analyze) does not report any problems on my PR.
[x] I updated pubspec.yaml, build.properties, and ios/Podfile with an appropriate new version according to the [pub versioning philosophy].
[x] I updated CHANGELOG.md to add a description of the change.
[x] I am authorized to release this code under the MIT License and agree to do so

UI Change
Does your PR affect any UI screens?

[x] Yes, the relevant screens are included below:

animated_align.json
animated_container.json
animated_cross_fade.json
animated_default_text_style.json
animated_padding.json
animated_physical_model.json
animated_positioned.json
animated_positioned_directional.json
animated_size.json
animated_switcher.json
animated_theme.json
directionality.json
tween_animation.json
main.dart by adding all the previous examples.

Wow, thanks!


Fix BadRequestData exception raised when trying to expand entity payload received in entity events

The exception is raised since the entity payload misses a context.


first draft of operator_position functionality

Added verification of operator_position option.  Throws error if
invalid.
Possible operator_position options are:

undefined (use default behavior)
'before_newline'
'after_newline'
'preserve_newline'
'remove_newline'

Moved some code around in handle_operator in order to separate
conflicting logic when handling operators.
Modified allow_wrap_or_preserved_newline to handle operator
preserved newlines as well.
Fixed a few boolean variable assignments within if statements.
Though not always the case, the expression within the if can
be assigned directly to the variable
Added -> built tests
Within the tests, added an 'inputlib.js' file which holds common
input arrays.  This was necessary due to the matrices not working
cleanly with default behavior.
It is worth noting that although the python test file was modified,
this is a result of the built test and not actually providing
a python port for this option (yet).

So instead of figuring out the redundant indentation logic and attempt to condense the tests via matrices, I just made it easy on myself and separated the tests.  I added tests for each combination of operator_position and preserve_newlines, meaning there should be 10 new sections of tests total.
Also, as noted in the commit message - I'm expecting the python tests to fail until we verify the javascript functionality.  I can write the logic for python afterward.

Let me start by saying - great job.  Thanks for putting the full effort into this and I approve of everything you mentioned in your description, including not using matrix style for testing.
The size of the change is a little daunting - though I get that the test portions are largely boilerplate.
I'll take a look at the overall structure of the product code changes so that you are unblocked to do the python porting.  Then I'll look at the detailed tests to make sure everything is coming out the way it should.
Thanks again!

Yeah I thought the testing might have been overkill.  It was more my method of development than me testing the end result (start by ensuring all the operators worked properly -> the special colon case -> then a catch all).
Feel free to let me know what you want slimmed down and I'll do it.  However I will wait to hear the full product-changes feedback before working on it further.
Finally, this is my first sizable open source contribution - so don't hesitate to critique anything.  Normally I just fork and modify for my personal needs.

Okay, I'm done with my initial pass.
There's the design issue of needing to make the preserve_newlines <--> operator_position interaction consistent, and some code tweaks, but rather minimal.
Thanks again!

Sounds good.  I'll have time this weekend to take a look and make changes.

Wow - I think I see my misunderstanding.  The operator_position shouldn't do anything if preserve_newlines is false.  It's only positioning the operators that are placed next to a newline that the user wants preserved.  This is why you're mentioning the default behavior 'undefined' should equal 'before_newline'.
I took the option names far too literally.  I'll work on this now and have an update tonight hopefully.

@olsonpm - I've been away. It looks like you've been away too.  You wan to take another swing at this?

Not a problem.
I think where our ideas differ on how to handle default behavior (i.e. no option specified)
My understanding is that it should behave as it did before
It seems from your comment above that you before_newline to be the default behavior which will be a breaking change.

Hm, It's probably an error in understanding on my part.
I thought OPERATOR_POSITION.before_newline is the same as the current behavior.   I'll go look again.

Referring back to your sample code you can see a few subtle differences:
// Test code
var d = 1;
if (a === b
    && c) {
    d = ((c * everything / something_else)
            - a) %
        b;
    e
        += d;

} else if (!(complex && simple) ||
        (emotion && emotion.strong && emotion.name === 'happy')) {
    cryTearsOfJoy(many ||
        anOcean ||
        aRiver);
}

// Output: actual current
var d = 1;
if (a === b && c) {
    d = ((c * everything / something_else) - a) %
        b;
    e
        += d;

} else if (!(complex && simple) ||
    (emotion && emotion.strong && emotion.name === 'happy')) {
    cryTearsOfJoy(many ||
        anOcean ||
        aRiver);
}

// Output: operator_positioning = before_newline
var d = 1;
if (a === b &&
    c) {
    d = ((c * everything / something_else) -
            a) %
        b;
    e +=
        d;

} else if (!(complex && simple) ||
        (emotion && emotion.strong && emotion.name === 'happy')) {
    cryTearsOfJoy(many ||
        anOcean ||
        aRiver);
}

Ah, I see where we're out of sync.  In all the complexity, I didn't cover the core cases.
  opts.preserve_newlines = false;
  opts.operator_position = undefined;
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed
  bt('var res = a + b\n' +
      '    - c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed

  opts.operator_position = 'before_newline';
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed
  bt('var res = a + b\n' +
      '    - c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed

  opts.operator_position = 'after_newline';
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed
  bt('var res = a + b\n' +
      '    - c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed

  opts.operator_position = 'preserve_newline';
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed
  bt('var res = a + b\n' +
      '    - c / d * e % f',
      'var res = a + b - c / d * e % f');  // newline removed

  opts.preserve_newlines = true;
  opts.operator_position = undefined;
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f');  // unchanged

 // NOTE: Below is different from current behavior
  // Current behavior removes the newline, this is a breaking change, but this seems more consistent and correct.
  // However, if we want to be non-breaking, having the newline be removed here is also acceptable.
  bt('var res = a + b\n' +
      '    - c / d * e % f',
      'var res = a + b -\n' +
      '    c / d * e % f',);  // newline preserved, operator moved to before the newline

  opts.operator_position = 'before_newline';
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f');  // unchanged
  bt('var res = a + b\n' +
      '    - c / d * e % f',
      'var res = a + b -\n' +
      '    c / d * e % f',);  // newline preserved, operator moved to before the newline

  opts.operator_position = 'after_newline';
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f',
      'var res = a + b\n' +
      '    - c / d * e % f',);  // newline preserved, operator moved to after the newline
  bt('var res = a + b\n' +
      '    - c / d * e % f');  // unchanged

  opts.operator_position = 'preserve_newline';
  bt('var res = a + b - c / d * e % f');   // unchanged
  bt('var res = a + b -\n' +
      '    c / d * e % f');  // unchanged
  bt('var res = a + b\n' +
      '    - c / d * e % f');  // unchanged

What you have forces newlines after all operators even when there were none. What we want is allow newlines in a specific position for operators and optionally move the newline if the code has it in on the wrong side.

No problem - I added to the confusion by initially coding it that way.  I still have my branch so I can take a look at it this weekend and get back with an up-to-date PR.

Let's do the breaking change I showed above.
It is different, but I'd argue that it is also more correct.  Right now the beautifier punishes people for putting newlines on the wrong side of operators - it looses their intended formatting.  The "break" is that is it will try to preserve their formatting.  I'm okay with that.

I'm on board, and definitely agree it's more correct.

Just an fyi - I didn't find time this weekend unfortunately.  I am motivated to get this done though and will give an update by mid next week

No problem.  I (obviously) know how it goes.   Thanks!

@bitwiseman - Took a good stab at this tonight, ran into a case I'm unsure about:
When operator_position is set to remove_newline What should the following resolve to?
var a = b

=== c;

The answer depends on whether you want preserve_newlines or operator_position to take precedence.  This if statement would need to check for opt.operator_position !== OPERATOR_POSITION.remove_newline if we want the newlines to collapse in the above case.
I'll bump this in two weeks if I don't hear back.

This is why I didn't include remove_newline as an option in my list above.  :smile:
I figured preserve_newlines would take precedence: if set to false existing newlines are ignored, if set to true the behavior can be customized.
Unless you feel we really need remove_newline, I'd drop it.  It seems like something most people would not want - "keep newlines in general, but remove them when they occur before or after an operator"?

Ah - that's what I get for skimming.  Thanks much for the clarification.

@bitwiseman - I feel pretty good about this second draft.  Ensuring the operator position logic only applies when preserve_newlines is true simplifies everything, especially tests.
Honestly I was very surprised our breaking change didn't cause a bunch of tests to break.  I was expecting to have to go through and fix a ton of them.
By the way, instead of looking at the diff between the first -> second draft, it may make more sense to diff master -> second draft.

@olsonpm - any update?  I'm driving towards releasing 1.6.0 and I'd love to include this.

Okay, I'm pushing this out of 1.6.0.

I believe the logic you asked for is now implemented.  preserve_newline should be untouched.

This looks great!  Just need the python port and update to cli and we're good to merge.

Take a look at
https://github.com/beautify-web/js-beautify/tree/feature/add-operator-position

Just dug through the javascript version now and it looks good.  Just to make sure I didn't miss anything, you basically slimmed down logic - turned the cli option to use a dash instead of an underscore, which necessitated the Object.values polyfill instead of keys.  And then you re-organized some of the variables into more appropriate spots.
Thanks for catching the punctuation <--> positional-operators overlap.  Didn't think of that.

Re python: Actually. You should be able to read it mostly 1-to-1 from the javascript.

*Rebasing and cleaning up the commit message now.
If there's anything about my commit message you want changed, it would be easiest if you just create another pull request and modify it yourself to save back-and-forth.  I'm not concerned about having my name on anything.

I'm not concerned about the message.

Alright - I looked at the readme and couldn't figure out why some cli options were left out so I'll leave it be for now.

The readme being out of date is totally possible. But unless it applies to this change getting it up to date is not needed.

Ah I see I didn't update the help message in python.  Meh, I'll get that in a minute.

Gotcha - and yeah I just thought updating the readme might make more sense in a separate commit to master.

@olsonpm  - thanks again for all your hard work on this. :+1:

@bitwiseman what is the jsbeautifyrc option for this feature?

@NerdPad - what version are you using?

I am using v1.6.2. Here is the output that I am getting:
Source:
const result = true
    ? true
    : false;

After format:
const result = true ? true : false;

.jsbeautifyrc
{
    "indent_size": 4,
    "indent_char": " ",
    "eol": "\n",
    "indent_level": 0,
    "indent_with_tabs": false,
    "preserve_newlines": true,
    "max_preserve_newlines": 2,
    "jslint_happy": false,
    "space_after_anon_function": false,
    "brace_style": "collapse-preserve-inline",
    "keep_array_indentation": false,
    "keep_function_indentation": false,
    "space_before_conditional": true,
    "break_chained_methods": false,
    "eval_code": false,
    "unescape_strings": false,
    "wrap_line_length": 0,
    "wrap_attributes": "auto",
    "wrap_attributes_indent_size": 4,
    "end_with_newline": true,
    "operator_position": "preserve-newline"
}

Hey thanks much - I'll take a look at this tonight when I'm back from work.

@NerdPad - 1.6.2 doesn't have this functionality.  Please let me know what led you to believe this was implemented

@olsonpm I saw the commit was merged, so I assumed it was available in the latest version. What I didn't check was the release date of 1.6.2. Sorry about that.
Do we know when this patch will get in the main release?

Not a problem - I only asked in case we did anything wrong.
and @bitwiseman will be able to answer that question.

Doh, yeah, it went in just after 1.6.2.  I need to get 1.6.3 published. :smile:

@bitwiseman Please publish 1.6.3. :)

Thanks for the reminder. 😄

Done! Enjoy!


Python3 compatibility for Benchmarks

Three issues addressed here:

Dependency on dictionary iteration order

CharacterProperties.swift.gyb iterated a dictionary to produce its output.
Changed this to a list of tuples to ensure the order is predictable.
Regenerated CharacterProperties.swift to match the new order.

Python3 map returns an iterator, not a list

Changed a bunch of map calls to list(map(...)) to ensure the result is a list

Python3 int() expects a string, won't accept a list of characters

Added a concatenation step that is effectively a no-op on Python2

@swift-ci Please smoke test

@swift-ci Please benchmark

Performance: -O

Regression
OLD
NEW
DELTA
RATIO

FlattenListFlatMap
3679
4526
+23.0%
0.81x (?)

Code size: -O
Performance: -Osize

Improvement
OLD
NEW
DELTA
RATIO

ObjectiveCBridgeStubToNSDate2
360
330
-8.3%
1.09x (?)

ObjectiveCBridgeStubToNSDateRef
2380
2200
-7.6%
1.08x (?)

Code size: -Osize
Performance: -Onone
Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: 6-Core Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

@swift-ci Please test

@swift-ci Python lint


i cant select the text in collection description / or click on the link

What happened?
i cant select the text in collection description / or click on the link

Please reproduce in steps
/
Expected Behavior
/
What browsers are you seeing the problem on?
Chrome
Are you logged in?
Yes
At which address did you encounter bug?
nft.kodadot.xyz / kodadot.xyz
Screenshots
/
Relevant log output
/

Code of Conduct

[X] I agree to follow this project's Code of Conduct

New UI will fix this.

@onerawartist fyi, if you want to display link in collection description you can use Markdown link

Fix text selection in collection description


Fix all warnings and lints.

Mostly removing extern crate and replacing struct field initialization Foo { bar: bar } with the Foo { bar } shorthand, but there are a few other changes simplifying logic and removing inefficiencies.
To test, run cargo +stable clippy --all-features

ugh, rustfmt went to town on the generated protobuf code... so yeah just ignore all that when reviewing this

I'd hold off merging this, there is/was some initial push towards refactoring the entire codebase to more modern (and maintainable?) Rust. IMO this will add a lot of churn at this moment, making that traceability for that task a bit more tedious..
But then again, I've a bit busy these days so finish up the refactor..

I'm in the process of building with --locked and also pushing a new release to crates & master. Once that is done we'll at least have a known good copy and can start refactoring the dev branch as necessary.

Have finished releasing v0.1.2 so refactoring can go ahead.

I'd hold off merging this, there is/was some initial push towards refactoring the entire codebase to more modern (and maintainable?) Rust.

In my experience, the first step in any refactor should be to get the codebase clean (no warnings, all tests pass, etc.) so you have more confidence your refactoring didn't introduce any more issues. Also it breaks things up into a smaller "fix the warnings" commit that doesn't move anything around or make any externally visible changes, separate from whatever larger refactoring changes you want to do. But I guess it's up to you how you want to move forward.

This is now outdated, closing

This is now outdated, closing


Quite a lot of changes. Especially to ContextMenu.

Providing Skin System for ContextMenu, providing global ContextMenuProvider and SkinRenderers. Including 2 default skins Classic and Minecraft.
ContextMenuScreen factory used for the same above changes.
Various code improvements and test features.
New widget anchoring system and actually loading from a backup file if found.
More changes and a full reformat.
Supports 1.21.1
The DynamicHudTest.java class is now automatically added for integration if the argument --dynamicHudTest true is passed while running Minecraft in program arguments.

TODO:

[ ] Widget Tray :- The idea is that a user can discard/remove widgets from the screen temporarily and place them in a "tray" to be used later. So, the tray acts as a store conveyer to remove access unneeded widgets for the user.
[ ] New Modern Skin Renderer.
[ ] Add DynamicHUD YACL config screen to the game's options screen and remove Mod-Menu Integration (optional).
[ ] More customization to widgets themselves for the user (like the background color when in the editor).
[ ] Fix a few more issues regarding screen resize and widgets.
[ ] Add more versatile HUD rendering.
[ ] Add more usefull and fast Renderer2D functions.
[ ] More stuff to add in this TODO

This wont be merged until some of the TODO have been completed.


[AVM Module Issue]: avm/res/network/front-door-web-application-firewall-policy old API version and missing types

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Issue Type?
Bug
Module Name
avm/res/network/front-door-web-application-firewall-policy
(Optional) Module Version
No response
Description
The API version for avm/res/network/front-door-web-application-firewall-policy is old, resulting in deployment error when configuring setting the javascriptChallengeExpirationInMinutes policy setting.
(Optional) Correlation Id
No response

Hey @PaulJohnston88,
it has been a while. Could you please triage the issue and give a rough estimate when you may or may not be able to address it? 💪
To triage the issue, you either close it (if it does not make sense to you), or you remove the 'Needs Triage' & 'Reponse' labels and add a commment 🚀


Add dependencies in the setup.py

Update the setup.py to define which package are dependencies to work with the toolbox library

Resolvido por b4a9995

English
Solved by b4a9995


Videos not sending. NOT BANNED.

Sending messages with bluebubbles works perfectly, except for videos. Regardless of the fact I have private API features enabled, followed steps correctly and everything, videos won't send at all on my vm Mac. I reset and created a brand new vm multiple times to no avail. I have Ethernet connected to reduce Internet lag as well. I attached an image of the blue progress bar when sending videos, which eventually says not delivered. Please help , thank you !
Created a post initially but it was closed, sorry I was out of town without cell connection.

I'm having the same issue, but only with some videos, even though the containers are the same (*.mp4), and I was wondering if it had to do with size of the video. The video that just now failed is exactly 6.0 MB. I get the blue status sending bar that never finishes and ultimately ends in a "failed to send". I re-encoded the video a few times to see if iMessage was being finicky about format. If I get anywhere with it, I'll post back.

Followup: Had the same issue with another video. I think it's because iMessage won't send a video that Quicktime on Monterey can't play. The first option I tried above (using ffmpeg to re-encode the video to audio codec aac and video codec libxvid) resulted in a video that successfully sent.  I didn't have to add text to the attachment.


Support for HTTP cookies

Hi Oliver,
I would like to propose an enhancement: Optional cookie management for HTTP sources (disabled by default).
Background: We have to integrate a proprietary WMS as source, which has difficulties if clients do not support HTTP session management through cookies. In other words: The client (mapproxy) has to accept and pass back session cookies.
I would be glad if you would agree to support such a scenario with MapProxy and accept this contribution.
So: Could you please review the changes? Important: I know almost nothing about python, so please let me know if something is not Ok. Luckily it is quite easy to make MapProxy support cookies.
I can also offer to adjust the documentation. But before I start working on the docs, please let me know if you can image to accept this.
With best regards,
Andreas

Hi Oliver, are you still willing to accept this pull request?
I'd be rather glad...
Please let me know if anything is missing.
Best regards, Andreas

What is the status of this? Could this be brought back into a mergeable state by rebasing on current master and fixing up the docs?

closing due to inactivity

Continued on https://github.com/mapproxy/mapproxy/pull/482


Cosmetic issue with PLAYLIST_PREFIX

If I set
- PLAYLIST_PREFIX="Z - "
playlists will appear as
"Z - "Queen Recommendations
If I set it to
- PLAYLIST_PREFIX=Z  (with a space after Z)
playlists will appear as
ZQueen Recommendations
Quotation marks are ugly, but also a space after the prefix is needed.

Very strange. in my local env that doesn't happen

Maybe it is something related to Docker environment, gonna sort this out.

Maybe it's Portainer again. I always deploy from there.

Maybe it's Portainer again. I always deploy from there.

Don't worry, i am installing portainer now to try to replicate this, I am sure I'll find a solution.

it's not portainer, i have this too. environments are set directly in the compose stack. without the quotaion marks the space get's ignored.

it's not portainer, i have this too. environments are set directly in the compose stack. without the quotaion marks the space get's ignored.

It is docker compose fault.
Now the correct way to set the variable is
PLAYLIST_PREFIX="Prefix - "
I reccomend you to not use special chars like "'myplaylist' -01 '" in this variable, or something like ""Z PLAYLISTS" -"
releasing soon.

Resolved in Release v0.2.0
Please reopen this issue if you still encounter errors


Animations: Added emoji counter update animation (non-optimistic)

Implementation waits for network response before updating UI. Optimistic version in works at #485

Automated message from Dropbox CLA bot
@nashvail, it looks like you've already signed the Dropbox CLA. Thanks!

Coverage decreased (-0.5%) to 56.582% when pulling d4943ce25a35e30dae80de5d4bbad9a98fea9174 on nashvail:emoji-animation-nonoptimistic into bf5348e625efe9f67c97207440e0996fca706dbe on zulip:master.


Still not working in RC5 because of wrong dependencies

for some reason ng2-datepicker uses forms-deprecated. It is not clear form it's dependencies (which are bad defined by not using a version but github:angular/common-builds etc)
So when one is bootstrapping the app by
platformBrowserDynamic().bootstrapModule(AppModule);
the app won't start because of the exception below. After removing the Datepicker directive in my component, the application loads again (but without the datepicker..)
// ---------------------------------------------------------------------------------------
import {Component, OnInit, Input, Output, EventEmitter} from '@angular/core';
import {TranslatePipe} from 'ng2-translate/ng2-translate';
import {
  FormBuilder,
  FormGroup,
  FormControl,
  Validators
} from '@angular/forms';
import {DatePicker} from "ng2-datepicker";
import {FirmwareService} from "../../service/firmware.service";
import {FirmwareUploadModel} from "../../model/firmware.model";
import {ValidationService} from "../../service/validation.service";
import {ControlMessagesComponent} from "../../control-messages/control-messages.component";

@Component({
  moduleId: module.id,
  selector: 'app-firmware-modal',
  templateUrl: 'modal.component.html',
  directives: [DatePicker, ControlMessagesComponent],
  providers: [FirmwareService],
  pipes: [TranslatePipe]
})
export class MyComp ..

// ---------------------------------------------------------------------------------------

EXCEPTION: Error: Uncaught (in promise): EXCEPTION: Error in http://localhost:4200/app/my-comp/my-comp.component.html:65:20
ORIGINAL EXCEPTION: No value accessor for form control with unspecified name
ORIGINAL STACKTRACE:
BaseException@http://localhost:4200/vendor/@angular/common/src/facade/exceptions.js:27:23
_throwError@http://localhost:4200/vendor/@angular/common/src/forms-deprecated/directives/shared.js:64:11

Hey @juwil, this looks like a problem with your configuration, not the module. If you need any help configuring this or a sample config let me know.

i'm getting the same error.  i'm using rc5.  can you include a demo.html page?

This should be resolved; https://github.com/jkuri/ng2-datepicker/commit/41426ab95add6b84e9fc93d102c6d729b4586c90


Fix pointer problem

Fixes the problem reported in CosmWasm/wasmd#2011

@Mergifyio backport release/1.5 release/2.0 release/2.1


Add 1170 - Counting Perfect BST(english and bangla tutorial)

[x] Bangla Tutorial
[ ] English Tutorial

@Tahanima hi, should I start working or wait for assignment?

You created the issue, if you want to work on it you should assign yourself and start working.

Hi,
Are you working on this?

Hi, i am working on this problem . I will make a pull request soon enough.

I saw that you added Bangla tutorial, could you add English one as well?

I will. Just need some time to work on that. Thanks!


Add Parquet-Exporter helm chart

Moved from https://github.com/opencost/opencost-parquet-exporter/pull/10
Hey everyone,
I've created a first draft of a helm chart for the parquet-exporter. I do not have a setup where I can use AWS as of now, therefore the actual push to a S3 bucket is not yet tested. However, I was able to install the chart and run the cronjob in my minikube.
Used helm-docs to create a rough documentation around the chart
The chart itself is relatively straight forward I think
Let me know what you think!

I will review this PR after the change to the parquet export repo is merged.
we cannot merge this PR before this is done.


Be stricter when parsing zone files

DO NOT MERGE: This is still very much a work in progress.
This is an attempt to make the ZoneParser & zlexer stricter. Currently they're unintentionally quite permissive. For example both of the following records parse despite being invalid:
example.com. 42 IN SOA ns1.example.com."hostmaster.example.com."1 86400 60 86400 3600
nl. IN NSEC3PARAM 1 0 5""

This also has the benefit of eliminating the ZoneParser state machine which leaves much simpler and easier to understand code.
This still has lots of rough edges and corner cases that need to be addressed, but it should be ready for a least a curious once over.

Codecov Report

Merging #879 into master will decrease coverage by 0.2%.
The diff coverage is 67.81%.

@@            Coverage Diff             @@
##           master     #879      +/-   ##
==========================================
- Coverage   57.33%   57.13%   -0.21%
==========================================
  Files          41       41
  Lines       10577    10602      +25
==========================================
- Hits         6064     6057       -7
- Misses       3425     3441      +16
- Partials     1088     1104      +16

Impacted Files
Coverage Δ

generate.go
75.8% <0%> (-1.43%)
:arrow_down:

privaterr.go
63.38% <63.63%> (-3.29%)
:arrow_down:

scan_rr.go
67.44% <67.79%> (-0.97%)
:arrow_down:

scan.go
80.29% <68.66%> (-1.05%)
:arrow_down:

server.go
66.19% <0%> (+0.23%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2533d75...411f891. Read the comment docs.

This should probably wait for #793. After that issue is solved the scan_re.go will be cleaner and simpler to reason about and change.

I have no time to work on this right now so I'm going to close it for the time being. Hopefully I can resurrect this eventually. I happen to really like the ZoneParser.Next changes included.


use fast-gaussian-rasterization in gaussian-splatting

Great works!
Can I use it directly in gaussian-splatting？I tried but failed😿. Maybe I need to modify the training part of the code

Hi, sorry for the delayed reply.
Yes, you should be able to use it for any pre-trained model for 3DGS. However, there's a known issue with the mismatched culling function of ours v.s. the original, as mentioned here. I plan on matching up the culling function lately. Or you can simply replace the OpenCV to OpenGL camera parameter conversion function with ours that maps the depth to [-1, 1].
def convert_to_gaussian_camera(K: torch.Tensor,
                               R: torch.Tensor,
                               T: torch.Tensor,
                               H: torch.Tensor,
                               W: torch.Tensor,
                               n: torch.Tensor,
                               f: torch.Tensor,
                               cpu_K: torch.Tensor,
                               cpu_R: torch.Tensor,
                               cpu_T: torch.Tensor,
                               cpu_H: int,
                               cpu_W: int,
                               cpu_n: float = 0.01,
                               cpu_f: float = 100.,
                               ):
    output = dotdict()

    output.image_height = cpu_H
    output.image_width = cpu_W

    FoVx = focal2fov(cpu_K[0, 0].cpu(), cpu_W.cpu())  # MARK: MIGHT SYNC IN DIST TRAINING, WHY?
    FoVy = focal2fov(cpu_K[1, 1].cpu(), cpu_H.cpu())  # MARK: MIGHT SYNC IN DIST TRAINING, WHY?

    # Use .float() to avoid AMP issues
    output.world_view_transform = getWorld2View(R, T).transpose(0, 1).float()  # this is now to be right multiplied
    output.projection_matrix = getProjectionMatrix(K, H, W, n, f).transpose(0, 1).float()  # this is now to be right multiplied
    output.full_proj_transform = torch.matmul(output.world_view_transform, output.projection_matrix).float()   # 4, 4
    output.camera_center = (-R.mT @ T)[..., 0].float()  # B, 3, 1 -> 3,

    # Set up rasterization configuration
    output.tanfovx = np.tan(FoVx * 0.5)
    output.tanfovy = np.tan(FoVy * 0.5)

    return output

def convert_to_cpu_gaussian_camera(K: torch.Tensor,
                                   R: torch.Tensor,
                                   T: torch.Tensor,
                                   H: torch.Tensor,
                                   W: torch.Tensor,
                                   n: torch.Tensor,
                                   f: torch.Tensor,
                                   cpu_K: torch.Tensor,
                                   cpu_R: torch.Tensor,
                                   cpu_T: torch.Tensor,
                                   cpu_H: int,
                                   cpu_W: int,
                                   cpu_n: float = 0.01,
                                   cpu_f: float = 100.,
                                   ):
    output = dotdict()

    output.image_height = cpu_H
    output.image_width = cpu_W

    FoVx = focal2fov(cpu_K[0, 0].cpu(), cpu_W.cpu())  # MARK: MIGHT SYNC IN DIST TRAINING, WHY?
    FoVy = focal2fov(cpu_K[1, 1].cpu(), cpu_H.cpu())  # MARK: MIGHT SYNC IN DIST TRAINING, WHY?

    # Use .float() to avoid AMP issues
    output.world_view_transform = getWorld2View(cpu_R, cpu_T).transpose(0, 1).float()  # this is now to be right multiplied
    output.projection_matrix = getProjectionMatrix(cpu_K, cpu_H, cpu_W, cpu_n, cpu_f).transpose(0, 1).float()  # this is now to be right multiplied
    output.full_proj_transform = torch.matmul(output.world_view_transform, output.projection_matrix).float()   # 4, 4
    output.camera_center = (-cpu_R.mT @ cpu_T)[..., 0].float()  # B, 3, 1 -> 3,

    # Set up rasterization configuration
    output.tanfovx = np.tan(FoVx * 0.5)
    output.tanfovy = np.tan(FoVy * 0.5)

    return output

Thanks reply! I will try this

Hello, I'm having the same issue. What is the error your code is experiencing?


Missing license

Could you please add some license to the project? Is it allowed to be used in open-source software?

Sorry, this is a duplicate


Links in documentation pointing to source code are broken

Many links that intend to point to the actual source code of contracts in the Github repo are broken.
As an example, in https://docs.zeppelinos.org/docs/Initializable.html, the link in Source: Initializable.sol redirects to: git+https://github.com/zeppelinos/zos/blob/v2.0.1/contracts/Initializable.sol.
Removing the git+ does not fix this issue, since the path to the contract is wrong. In this case, it should be https://github.com/zeppelinos/zos/blob/v2.0.1/packages/lib/contracts/Initializable.sol

This seems to be a problem with solidity-docgen not prepared to handle multi-package repos like zos: https://github.com/OpenZeppelin/solidity-docgen/blob/master/src/components/document.js#L97

Submitted a fix for solidity-docgen. As soon as it's merged, updating the dependency and regenerating the docs should fix this issue.


PyPI upload failure

Not sure what is causing this error to upload the package. I know that they recently changed things to support attestations, maybe we need to implement that to get it working?
https://github.com/brainiak/brainiak/actions/runs/11875573414/job/33094720463

Any chance that we need to do this on the brainiak pypi account:
https://docs.pypi.org/trusted-publishers/adding-a-publisher/

It was the indeed a PyPI publisher issue, thanks for spotting it.