Request for comments: service.type and service.distro Resource attributes

Problem Description
service.name Resource attribute is currently defined as the "Logical name of the service". The expectation is that service.name will be set by the operator of the service to a value that describes the role of the service in the overall observable set of entities the operator has (within a `service.namespace).
Otel Collector sets service.name by default to be the name of the executable (e.g. otelcorecol or otelcontribcol).
Collector's service.name can be overridden by the operator using service.telemetry.resource setting of Collector's config file. This is typically expected in any non-trivial infrastructure where the same Collector executable can be used as a locally running agent on a host, as a standalone gateway that serves as an intermediary between agents and the backends, as part of Kubernetes operator, etc. The roles in these cases are sufficiently different to warrant different logical names.
However, there is currently no semantic convention for an attribute that specifies the type of a service that may have different logical roles when used in different places in the infrastructure, yet be identically produced, i.e. be the exact same executable. The executable file name to some extent can serve that purpose but nothing prevents different service types from having the same executable file name, it has poor uniqueness guarantees.
This issue talk a bit more about why we would want to have the type of an agent (Otel Collector in our case) to be a well-defined semantic convention.
This issue shows how the agent type would be useful in the context of agent management. The issue talks about how it is important to tie agent's own telemetry's Resource to the attributes that identify that agent in the context of the OpAMP protocol.
Proposal
This is a request for comments for adding the following Recommended, experimental Resource semantic conventions:

service.type - an FQDN that uniquely identifies the type of the service, e.g. io.opentelemetry.collector, io.redis, etc. Unlike (service.namespace,service.name,service.instance.id) triplet the (service.namespace,service.type,service.instance.id) triplet is not guaranteed to be globally unique.
service.distro - an FQDN that uniquely identifies the distribution of the service. A number of distributions can belong to the same service.type. For example OpenTelemetry Collector has multiple known distributions, e.g. io.opentelemetry.collector-contrib, com.splunk.opentelemetry-collector, com.amazon.adot-collector, etc. [as an alternate instead of reverse company FQDN we may also require service.distro to point to the source code repo of the distro (e.g. git/github repo) for OSS agents].

Note that having a separate service.type allows OpAMP if wanted by the operator to manage the same type of agents in a similar way even though their service.name values may be different due to different logical roles they have.
Another example with NGINX: service.type will be set to com.nginx by NGINX developers, while service.name is set to "api-gateway" by the operator, denoting the logical role that the particular NGINX deployment serves in this particular system.
I am looking for comments and thoughts on this proposal before I submit a PR.

@open-telemetry/collector-approvers @open-telemetry/collector-contrib-approvers I would also like your opinion on this proposal.

While I see the appeal for service.type, I'm not too keen on service.distro. I can't think of a workload that has different distributions other than the Collector. Do you know other examples that help validate that this is needed beyond the collector?
In any case, I do believe that the collector has to provide the distribution as part of the resource attributes for its own telemetry.

The main intent of semantic conventions is defining the shape of telemetry data, and I have some mixed feelings when I see them used in modelling parts of protocol specifications.

Do you know other examples that help validate that this is needed beyond the collector?

That's also something I'm wondering about: do we see value in putting those attributes on telemetry data? Or is the primary intent to satisfy requirements stemming from coupling the OpAmp spec with semantic conventions?

I can't think of a workload that has different distributions other than the Collector. Do you know other examples that help validate that this is needed beyond the collector?

@jpkrohling here's an example from @tigrannajaryan:

For example PostgreSQL has a bunch of forks and derived databases which this attribute can indicate.

https://github.com/open-telemetry/semantic-conventions/issues/396#issuecomment-1759959883
This is not to say I agree with this reasoning. Wouldn't all these different databases have a different service.type, if that existed?
I'm a bit worried that service.type is too vague. Different people might want to set it to different things in different contexts.
If this is needed for OpAMP (as this comment suggests: https://github.com/open-telemetry/semantic-conventions/issues/396#issuecomment-1800291307), why not intruduce an OpAMP-specific attribute like opamp.agent.type as proposed by @jack-berg here: https://github.com/open-telemetry/opamp-spec/issues/131#issuecomment-1263884514?

Wouldn't all these different databases have a different service.type, if that existed?

Yes. In the same vein, Elasticsearch vs. OpenSearch. As a user, I would likely prefer to have each fork on its own service.type. Collector core vs. contrib are not forks, they really are different distributions.

The main intent of semantic conventions is defining the shape of telemetry data, and I have some mixed feelings when I see them used in modelling parts of protocol specifications.

To be clear: the only reason OpAMP is concerned with this is to direct the Collector to report useful telemetry about itself and to be identified in a way that is useful for the end user. There is no other use for agent type in any other capabilities that OpAMP supports.

That's also something I'm wondering about: do we see value in putting those attributes on telemetry data? Or is the primary intent to satisfy requirements stemming from coupling the OpAmp spec with semantic conventions?

I think there is value in putting this in telemetry. If I am running a cache that I give a service name "foo" it is likely useful for me to also know that that cache is actually a Redis instance.
If we don't see the value of this then we should not add a service.type. Instead we can go with agent.type and other agent-specific conventions.

If this is needed for OpAMP (as this comment suggests: #396 (comment)), why not intruduce an OpAMP-specific attribute like opamp.agent.type as proposed by @jack-berg here: open-telemetry/opamp-spec#131 (comment)?

Yes, that's the alternate if we don't believe that service.type is generally useful for all services.

To be clear: the only reason OpAMP is concerned with this is to direct the Collector to report useful telemetry about itself and to be identified in a way that is useful for the end user. There is no other use for agent type in any other capabilities that OpAMP supports.

I was referring to the identification part, where semantic convention attributes seem to be part of the OpAMP spec.
From a non-OpAMP point of view, I find service.distro quite confusing. For collectors I think this would overlap with  telemetry.distro.name, for non-collectors or non-agents I'm not sure what would go into this attribute.

I think there is value in putting this in telemetry. If I am running a cache that I give a service name "foo" it is likely useful for me to also know that that cache is actually a Redis instance.

That's definitely a valid use case (for service.type), as currently the only other way to achieve this is deducing the "type" from other (resource) attributes. Do you think it makes sense beyond third-party OTel-enabled services (e. g. redis, nginx, kafka, collector)?

I was referring to the identification part, where semantic convention attributes seem to be part of the OpAMP spec.

Yes, we are in agreement here, that's what I meant by "to be identified in a way that is useful for the end user".

From a non-OpAMP point of view, I find service.distro quite confusing.

This is probably somewhat unique to Otel Collector. If we look at Postgres for example there is a ton of Postgres-based databases, but I am not sure I would call them a "distro", perhaps they would be their own "service types".

Do you think it makes sense beyond third-party OTel-enabled services (e. g. redis, nginx, kafka, collector)?

Probably of quite a limited use for the first-party services. Something developed and used internally in most cases has a well known internal name that is recorded in service.name, and which likely is enough. However, I can imagine in big organizations you may see a first-party service that is generic enough to be built once but deployed in many different places and roles to warrant a difference between the service.type and service.name (e.g. if an org implements their own storage system they may give it its own common service.type but use different service.names depending on what it is used for). This should be likely rare, but perhaps not impossible.

Envoy proxy can also be extended and built with custom extensions, essentially being another "distro". For ex, the Istio project uses it's own distro of Envoy (https://github.com/istio/proxy)

I support the two proposed new attributes. In this related work, https://github.com/open-telemetry/oteps/pull/238, I've tried to establish a uniqueness requirement for resource attributes of some kind. The solution proposed in this issue will help. We want to be sure that no two agents with identical attributes handle a pipeline, otherwise they appear to be double-counting the items that pass through them both. If we have semantic conventions that ensure pipelines have unique stage names, we can do more to automate pipeline monitoring.

My quick reaction to one comment is that I think the use case of OpAMP implies the use of service.role over service.type.
Specifically, the OTEL collector being used as a an agent vs. gateway that seems more like a "role" and not something a FQDN would address.

Specifically, the OTEL collector being used as a an agent vs. gateway that seems more like a "role" and not something a FQDN would address.

@jsuereth I would expect the role to be recorded in service.name. It is defined as "Logical name of the service", which seems to be a fine location for what one would call a "role".

To make the OpAMP use case clear here, assume we have an OpAMP client that works on the collector and reports the collectors config under the effective_config section. It's then implied that an OpAMP server who is able to read and unmarshal this effective configuration will want to do so by using the collectors unmarshaling logic. This is relatively straightforward, a collector OpAMP client connects to a collector OpAMP server.
As OpAMP is adopted by more agents, however, this is more complicated. Because OpAMP allows for a partial implementation, any agent type that implements the protocol can connect to our server, not just an OpAMP collector client. The next OpAMP client implementation is by the OpAMP bridge in the operator which reports on the Kubernetes configuration for a collector, this is decidedly different than our OpAMP collector client's effective configuration.
In the future, say an agent like Prometheus were to have an OpAMP client, our OpAMP server would want to be able to marshal this configuration using Prometheus' unmarshal logic.
All of this necessitates a type that an OpAMP server can switch on to know what type of client it's talking to. Without wading too much in to the discussion of this, I would posit that any collector will have the same structure of configuration, the difference being who built the image. To that end, I would recommend that any images that are functionally equivalent and share a configuration structure should have equivalent service.type, but differing service.distro which (to me) says who exactly built the image you're using. This would result in service.role  for the OpAMP use case having client and server which would allow an end user to understand how their OpAMP agents and servers are being run.
I'll tie it up in this example:
sequenceDiagram
    participant P as OpAMP-Client (Prometheus, vanilla)
    participant OTC as OpAMP-Client (OTC, vanilla)
    participant SOTC as OpAMP-Client (OTC, Splunk)
    participant OB as OpAMP-Client (Bridge, vanilla)
    participant OPS as OpAMP Server
    P->>+OPS: sends effective config
    OTC->>+OPS: sends effective config
    SOTC->>+OPS: sends effective config
    OB->>+OPS: sends effective config

    OPS-->>-OTC: Responds OK
    OPS-->>-SOTC: Responds OK
    OPS-->>-P: Responds OK
    OPS-->>-OB: Responds OK

We can see that each OpAMP client is able to report its role, (type, distro) which allows a single server to understand each request. All of this would be supported by the above proposal. Thank you @tigrannajaryan

yes – I don't think it's terribly useful for us outside of a reporting case where you could image the client and server emitting the same metric (connections for example) to an external telemetry vendor. Without the role, the vendor would receive
connections{service.type=OTC, service.distro=otel} 1 # from the client
connections{service.type=OTC, service.distro=otel} 1 # from the server

however with a role it would be more descriptive:
connections{service.type=OTC, service.distro=otel, service.role=client} 1
connections{service.type=OTC, service.distro=otel, service.role=server} 1

This example is admittedly contrived, and there's a better use case for collectors IMO.

The http connection metrics already have their own, distinct way of differentiating the client and server metrics. They have different metric names, there is no need for an attribute: https://github.com/open-telemetry/semantic-conventions/blob/main/docs/http/http-metrics.md

still, i think role would be useful here for the opamp supervisor case. The supervisor acts as a middleman and without the role the telemetry would look like this:
flowchart TD
    A[OpAMP Extension] -->|http| B(OpAMP Supervisor)
    B -->|http| C{OpAMP Server}

# from the opamp extension's client
http.client.request.duration{service.type=OTC, service.distro=otel} 1
# from the opamp supervisor's server
http.server.request.duration{service.type=OTC, service.distro=otel} 1
# from the opamp supervisor's client
http.client.request.duration{service.type=OTC, service.distro=otel} 1
# from the opamp server
http.server.request.duration{service.type=OTC, service.distro=otel} 10

Without role, it's unclear the relationship between these related metrics. In this example we have high latency in the opamp server, but it's unclear if that would be affecting the opamp extension or the opamp supervisor. Adding in the role would allow the telemetry to understand the relationship between these metrics:
# from the opamp extension's client
http.client.request.duration{service.type=OTC, service.distro=otel, service.role="extension"} 1
# from the opamp supervisor's server
http.server.request.duration{service.type=OTC, service.distro=otel, service.role="supervisor"} 1
# from the opamp supervisor's client
http.client.request.duration{service.type=OTC, service.distro=otel, service.role="supervisor"} 1
# from the opamp server
http.server.request.duration{service.type=OTC, service.distro=otel, service.role="server"} 10

still, i think role would be useful here for the opamp supervisor case. The supervisor acts as a middleman and without the role the telemetry would look like this:

I don't know what I'm talking about, but this looks to me like maybe an example of how metrics have less fidelity than traces. There are three resources in the diagram, but no mention of their identities in the metric attributes/labels.

To make the OpAMP use case clear here, assume we have an OpAMP client that works on the collector and reports the collectors config under the effective_config section.
The next OpAMP client implementation is by the OpAMP bridge in the operator which reports on the Kubernetes configuration for a collector, this is decidedly different than our OpAMP collector client's effective configuration.

If we look at Postgres for example there is a ton of Postgres-based databases, but I am not sure I would call them a "distro", perhaps they would be their own "service types".

I don't know what I'm talking about, but this sounds like OpAMP has a concept of what makes two things similar or different. Should the discussion here be about an attribute in an opamp namespace/domain?

Should the discussion here be about an attribute in an opamp namespace/domain?

That is the alternate approach if we decide that service.type is not generally useful for all services. I see some supporting voices that it can be useful so I am going to submit a PR that adds it and continue the discussion there.

All, the PR that adds service.type is created, but I and others have doubts that this is the right way. Please comment on the PR with arguments in favour or against it.

Closing this as "won't do". We will likely instead propose agent.* semantic conventions.


templates: Remove context argument from {{#tr}} block helper

It only had one nontrivial use, and it’s easily replaced using the builtin {{#with}} block helper.
Testing plan: Tested in dev server, with a focus on the message star tooltip.

Do we need to document use of the with builtin?  Otherwise lgtm.

It’s a standard Handlebars builtin and we already use it 20 times.

OK.  Merged, thanks @andersk!


test case to reproduce  "Wrong Transports: 1" message

input graph:
1 3
1 4
1 5
1 6
2 3
2 4
2 5
2 6
spark version: 2.1.0
the parms as flow:
{
w2vIter:	10,
w2vLr:	0.025,
w2vPartitions:	1,
w2vDim:	128,
w2vWindow:	10,
walkLength:	20,
numWalks:	10,
p:	1.0,
q:	1.0,
weighted:	true,
directed:	false,
input:	/user/kgalg/node2vec/graph/karate.edgelist,
output:	/user/kgalg/node2vec/randomwalk,
rddPartitions:	200,
singleOutput:	true,
partitioned:	true,
cmd:	randomwalk
}
running log:
randomwalk.log
Q1:  why does "Wrong Transports: 1" often appear in log file?
Q2: what triggers the "ERROR org.apache.spark.internal.Logging$class.logError" message in the output?
Q3: Are these two issues related to spark version? spark2.1.0 does work?

We notice the same thing: If we run the program in local mode, the result is correct. However, when you run the application in standalone mode, the following line:
https://github.com/data61/stellar-random-walk/blob/master/randomwalk/src/main/scala/au/csiro/data61/randomwalk/algorithm/VCutRandomWalk.scala#L112
results in empty GraphMap object outside the mapPartitionWithIndex method. Besides, manipulating an object inside each partition seems very strange. Is it correct?


Caching in SuperSet

I have the following query running in SQL Lab
(SELECT
        x.a
        x.b
        y.a
        y.b
   case( logic here )
     ... fetching other attributes from tableX and tableY
   FROM  tableX x
   join tableY y on
   x.id = y.id  ) AS TEMP_VIEW

Now i will creating charts based on. this query i.e will extract some of the attributes from the above query
For Example :
SuperSet will form the query like this :

Chart 1:
 Select x.a, count(*)
 from TEMP_VIEW
 Group By x.a

Chart 2:
 Select y.a, count(*)
 from TEMP_VIEW
 Group By y.a

// Other Charts and Filter based on TEMP_VIEW Attributes

Since all this charts will on one dashboard how will i make sure that TEMP_VIEW is executed only once when i load the dashboard  i.e i don't want to hit my data source for all the charts as the output will be same.
My Current Design is like this :
SuperSet -> Presto -> Big Query

If you have concern around performance / cost around that view, you may want to materialize it. Superset is not a database or compute engine, it can't store the output of your view and summarize against it.

Hey we had the same issue. We are storing the results of query in  a parquet file and then on that parquet file data, we are firing SQL Queries.


Incorrect optimized code for GuardFieldClass if final field is not yet initialized

The code generated in GuardFieldClassInstr::EmitNativeCode will be incorrect (logically a no-op) if the following conditions hold:

The instruction is compiled in optimized mode.
The referenced field is final (and therefore can participate in length guarding).
The field has not yet been initialized.

GuardFieldClassInstr::EmitNativeCode falls-through after the check if the check fails. However, the fall-through case is generated conditionally by three different blocks, none of which trigger if the above conditions are true.
@mraleph

Fixed by https://dart-review.googlesource.com/c/sdk/+/98329


Is to possible to add text inside a gauge ?

Hi,
Is it possible to add some text inside the gauge while creating it ? If not, can some one help me obtaining it using css. I have add a sample image which i created using paint.

It is possible to do this using Css tricks. Let me send you an example
tonight.
On Thu, Oct 15, 2015 at 8:15 AM Tarun<EMAIL_ADDRESS>wrote:

Hi,
Is it possible to add some text inside the gauge while creating it ? If
not, can some help me obtaining it using css. I have add a sample image
which i created using paint.
[image: gauge]
https://cloud.githubusercontent.com/assets/2683705/10518129/9975dd2c-737d-11e5-8728-2c278f9fe76e.jpg
—
Reply to this email directly or view it on GitHub
https://github.com/epochjs/epoch/issues/206.

Here is an example of how to do this with the "gauge-small" type chart:

    Text Here

Which results in this:

The key here is the top attribute. For this I simply took the height of the gauge-small class (135px) and subtracted 10 to get the desired result.
Each size for the gauge has its own pre-defined height, for the exact size for each type see the documentation on the Epoch Github Pages site.


server error

ı tried to change my user name .But ı received an error  message .Can you check it please?

Can you send error log?

Request Method:
POST

http://www.coogger.com/settings/profile

1.11.5

UnboundLocalError

local variable 'form' referenced before assignment

/home/coogger/coogger/cooggerapp/views/csettings.py in profile, line 17

/usr/local/bin/uwsgi

3.5.2

['/var/www',  '.',  '',  '/var/www',  '/home/coogger/coogger/lanet/lib/python3.5',  '/home/coogger/coogger/lanet/lib/python3.5/plat-x86_64-linux-gnu',  '/home/coogger/coogger/lanet/lib/python3.5/lib-dynload',  '/usr/lib/python3.5',  '/usr/lib/python3.5/plat-x86_64-linux-gnu',  '/home/coogger/coogger/lanet/lib/python3.5/site-packages',  '/home/coogger/coogger']

line 17 csettings.py form


Document the operators available for provisioner .spec.requirements

Description
How can the docs be improved?
It's not clear from https://karpenter.sh/preview/concepts/provisioners/ what operators are available for use,
The page has mention of In, NotIn and Gt but it does not mention Lt , or any other operator.
From example is unclear to me if there is a GtOrEqual  operator or something like that.
I search in website/content/en/v0.30 and there is no mention of Lt
The pkg/apis/crds/karpenter.sh_provisioners.yaml explictly mentions that the valid operators are In, NotIn, Exists, DoesNotExist, Gt and Lt
https://github.com/aws/karpenter/blob/cce5dbc8ca580f17772e7f532d258011f102d966/pkg/apis/crds/karpenter.sh_provisioners.yaml#L213-L214
But I think that should be made explicit in the documentation at least with a link to the pkg/apis/crds/karpenter.sh_provisioners.yaml  saying that the list of valid operator is there , or something like that.

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

We support all the upstream K8s operators mentioned here: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#operators
We can add a link to the upstream docs


Fix weird word order in chaining-if-else-statements Spanish translation

Fix order of name/adjective in Spanish translation of chaining-if-else-statements

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

Closed and Reopened this PR in an attempt to attempt to break out of the Travis build pending status.

Thanks!


Fix #4716

Fix issue #4716: fix the return value of Stream::write in case of failure.
Contributor agreement: https://developer.mbed.org/users/Nodraak/

PR on wrong repo


Compared to the original InfiniTAM, does this implemention need parameter adjustment?

I am wondering that since the original InfiniTAM works indoor and DynSLAM works outdoor, how to adjust the parameters so that InfiniTAM can work outdoor? The depth information is quite different in these two situations.

This depends on the sensor you are using, how your stereo rig is configured, etc.
Also, in many outdoor scenarios the motion tends to be faster than in indoor ones (e.g., in KITTI you can have very fast motion but only 10Hz inputs), so you'd need to use a different camera tracking method.
Do you have any details about the context in which you wish to make this work? Do you want to run DynSLAM on indoor data?

Thanks for your reply!
I want to use a stereo camera which outputs depth map in real time for outdoor reconstruction.
I employ VINS-MONO as the visual odometry, so the camera pose is well got.
Now the InfiniTAM can reconstruct indoor scenarios quite well, but when it comes to outdoor objects, the result is not that satisfactory. My stereo camera outputs depth up to 20m, and obviously the original InfiniTAM cannot fuse depth that far.
I knwo there are several parameters affecting the reconstrucion quality. The code defines a class called sceneParams which wraps the parameters related to the scene representation.  I tried several combinations of these parameters but cannot achieve a satisfactory result.
So how do DynSLAM adjust the parameters so that it can work outdoor? Do you have a set of parameters? If the parameters are suitable, maybe the problem comes from depth map.

Yes, it's likely an issue with how the depth is computed. Have you tried tweaking the calibration parameters, e.g., the stuff from the sample file here: https://github.com/AndreiBarsan/DynSLAM/blob/a7e885205de9bf8e40527c834398c63ad2f90641/itm-sample-calib-from-kitti-odometry-sequence-06.txt ?
This calibration is read here in the code: https://github.com/AndreiBarsan/DynSLAM/blob/a7e885205de9bf8e40527c834398c63ad2f90641/src/DynSLAM/DynSLAMGUI.cpp#L1162
The last line in the calibration file is very important. This thread could be helpful in explaining it: https://github.com/victorprad/InfiniTAM/issues/130

Closing this due to lack of activity. Please don't hesitate to reach out if you still have questions/problems!


[FEAT]Hotkey to improve accessibility while editing table

Enter to edit, and ctrl+enter to exit edit mode.
Up \ Down \ Left \ Right to move between cells like tab and shift+tab
Ctrl+C & Ctrl+V to copy or paste cell.

I've been working on this, but unfortunately I've encountered a bug in Svelte that has made this rather tricky. I'll continue to work on it though.

I've addressed most of this in 0.16.0, except using the arrow keys (that one's trickier), but I'll create a separate issue for that.


Out of date dependencies

Please update dependencies decompress and download to latest version because they require gulp-util which is depreciated.
Thanks!

https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5

The current version of download being used ^6.2.2 also has a sub-dependency of tunnel-agent, which contains a Memory Exposure vulnerability.
download > caw > tunnel-agent
https://nodesecurity.io/advisories/598

download < 11.8.5 also drags https://github.com/advisories/GHSA-pfrx-2q88-qq97

@kevva any updates on this?
is this project still maintained?


Add augmentations to training

Well, I'm not sure why I cannot change the target branch. @hdefazio are you able to make this target master?


Readme update

Update readme and make it more fun In this folder
Web Development
Reference: https://kushal997-das.github.io/Project-Guidance/

Update readme and make it more fun In this folder
Web Development
Reference: https://kushal997-das.github.io/Project-Guidance/

Any body please contribute here....to help us...

Update readme and make it more fun In this folder
Web Development
Reference: https://kushal997-das.github.io/Project-Guidance/

Any body please contribute here....to help us...

Hey I'd like to work on this can you provide me with notes for the README

Hey I'd like to work on this can you provide me with notes for the README

Hey I'd like to work on this can you provide me with notes for the README

There is no such notes see the desktop application folder for reference and do the same job in web development folder. I hope it make sense @JohnWesleyK

@Kushal997-das I've updated the web development README and here is the link to the PR related to that

Done!


ZoneParser: error on parsing an IPv6 address in an A record

And vice versa for IPv4 with AAAA.
The implementation of isIPv6 is inspired by https://github.com/golang/go/blob/e341bae08d75611adea6566c1d01c1e3a0de57f9/src/net/ip.go#L678-L681 .

Codecov Report

Merging #923 into master will increase coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #923      +/-   ##
==========================================
+ Coverage   55.09%   55.14%   +0.04%
==========================================
  Files          41       41
  Lines       10059    10061       +2
==========================================
+ Hits         5542     5548       +6
+ Misses       3489     3486       -3
+ Partials     1028     1027       -1

Impacted Files
Coverage Δ

scan_rr.go
66.33% <100%> (+0.05%)
:arrow_up:

msg.go
77.54% <0%> (+0.65%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e838e1e...8f11201. Read the comment docs.

On Fri, 8 Mar 2019, 10:06 Tom F<EMAIL_ADDRESS>wrote:

@tomfitzhenry commented on this pull request.
In scan_rr.go
https://github.com/miekg/dns/pull/923#discussion_r263725942:

@@ -133,7 +133,8 @@ func (rr *A) parse(c *zlexer, o, f string) *ParseError {
}

rr.A = net.ParseIP(l.token)

if rr.A == nil || l.err {

isIPv6 := strings.Contains(l.token, ":")

I considered that, but that causes test failures because
net.ParseIP("::FFFF:<IP_ADDRESS>").To4() is non-nil.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/miekg/dns/pull/923#discussion_r263725942, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAVkWxO_IU_dOKPAdI8S8YWMSyfvxTHZks5vUjZCgaJpZM4bkmsr
.


dataproc-initialization-actions prometheus.sh no longer in GCS

Running
gcloud dataproc clusters create <My_own_CLUSTER_NAME> \
    --initialization-actions gs://dataproc-initialization-actions/prometheus/prometheus.sh

as documented in
https://github.com/GoogleCloudPlatform/dataproc-initialization-actions/tree/master/prometheus
gave me the following error:
ERROR: (gcloud.dataproc.clusters.create) INVALID_ARGUMENT: Google Cloud Storage object does not exist 'gs://dataproc-initialization-actions/prometheus/prometheus.sh'.
@karth295
@szewi
I will try downloading and uploading the .sh file on Github to my GCP project GCS and call from there manually if the .sh file is not outdated

Thanks for the issue -- looks like the github repo and the GCS got out of sync. I've manually synced them again, so the Prometheus init action works now!


deps(http-proxy): update dashmap, twilight, metrics-exporter-prometheus

Update to latest twilight, use a dashmap version with the sanity fixes

$ cargo outdated -wR
twilight-http-proxy
================
Name                         Project  Compat  Latest  Kind    Platform
----                         -------  ------  ------  ----    --------
hyper-trust-dns              0.3.1    ---     0.4.2   Normal  ---
metrics-exporter-prometheus  0.8.0    ---     0.9.0   Normal  ---
twilight-http-ratelimiting   0.9.0    ---     0.11.0  Normal  ---

I suggest bumping hyper-trust-dns too

Could you pull the changes and update the API version in the readme to 10


Add option to install the self-contained binary version of pnpm

Related to #31
close #18
Introduce new input nodejs_bundled. If true, install @pnpm/exe
This new option is convenient for Node.js 14 + pnpm 8 or similar.

I have merged your commits into a new PR: https://github.com/pnpm/action-setup/pull/92

switch to #92 and close this PR


Get Electron build working

Since becoming maintainer of this project, and making some security updates, we have not been able to get the electron build working. This is why there has not been a release in a while.
This is no small undertaking, but very very important!

I gave this a shot, after few version package changes now the yarn release:mac is building successfully. Though when I click the app I get a message saying "Loading GraphQL Playground" and then blank screen. Will dig more into this, let me know if you have any suggestions @acao .

looks like progress! now you’ll need to open the dev tools in the electron app to see what’s up. there should be a process in the readme that describes how to run a watcher for both the webpack and electron build, might require two processes at once

Fixed few more errors, now need to fix this Uncaught Error: React is running in production mode, but dead code elimination has not been applied. Read how to correctly configure React for production: https://fb.me/react-perf-use-the-production-build (saw in console of electron app) , after this I think it should be good.

looking better, yeah! can you push what you have?

@acao I fixed the above error but still get the same broken UI like above. When I enter URL and click open, then console has the error TypeError: Cannot read property 'store' of null. I think that the bundle generated is missing it. I will dig into the webpack config generating the bundle (maybe something is missing there).
Do you have any suggestions?

it looks close! looks like it just needs the CSS loader to be reconfigured somehow, and we need to revisit how the redux store is being loaded, in case a breaking change in a redux dependency or some such impacted that

@acao got it to load the playground successfully now! Schema is getting fetched and docs tab is correct as well but I get this error Uncaught Error: Expected { __allowedLegacyNames: undefined, _queryType: Query, _mutationType: Mutation, ... } to be a GraphQL schema. in console when I start typing the query. The CSS part is because of the loader as you suggested, the styled-jsx-postcss/babel is missing from before I think it's because of that, trying to fix that.
Any suggestions for the schema error?

@acao got that error fixed, now playground is running queries as expected! Now just need to fix the CSS loading for the initial view, apart from that I think everything else is looking good now!
I have updated the PR for this.

@vardhanapoorv awesome work!

@acao I'm not able to get the current setup for postcss to work. I tried few ways but no success yet. Error from happypack loader Invalid PostCSS Plugin found: [0]
Not sure if changes will be needed to the fork of postcss-inherit that we are using. Should we have the CSS defined here itself since there isn't much?
Maybe I'm missing something very silly in the config. Any suggestions on how to go about this?

@vardhanapoorv good question, try getting rid of the postcss for now. not very important with the electron runtime

@acao I actually fixed it a bit but still some part of postcss (postcss-inherit I think) isn't getting applied. Anyway I have for now just added the CSS classes directly it looks good to me! so I have updated PR, can check the postcss thing later if needed. You can review it.

looking fantastic! i'll take a look here soon

@acao I had like to work on this issue.

@acao I had like to work on this issue.


⚠ ClamAV has degraded performance

In 461e9d0, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 441 ms

Resolved: ClamAV performance has improved in f689904.


Data Collection

Motivation

In order to work with the interviews in order to extract insights, we need to first store the raw interview data in a way that makes it accessible for people who will be working on the synthesis.

Considerations

Should all interviews be recorded?
Where and how should we store the videos?
How can we make it accessible to the people working on the research?
How can we keep the data GDPR compliant?
Can we publish an anonymized version of the interviews? And where?

Outcomes

A clear, GDPR compliant process, that allows use to store the raw interview data for synthesis.

Including @krook from CNCF staff - can the CNCF help us set up and use an interview and data management tool to best collect data (this issue) and synthesize (#552) our research?
Also, can you help us apply appropriate privacy and legal practices to the raw data and published reports? Thanks!

For those interested in interviewing from their own network:

Notify the PaaP leads via google form with the interviewees name, company name, and role, along with your name and email
Access the Platform Engineer script and Interview Protocol in github. Review Best Practises Guidelines.
Schedule the interview (email must include a GDPR blurb on the purpose of the research and data usage which will be found in the same github repo)
Conduct the interview
Submit the full transcript with interviewer notes (your immediate impressions while they're fresh in your mind) via google form.

For those interested in interviewing from the CNCF network:

Sign-up via google form. Provide level of user interview experience.
Depending on your level of experience, you'll be be the lead or secondary on the your first interview.
You'll be provided with an interviewee.
Schedule the interview (email must include a GDPR blurb on the purpose of the research and data usage which will be found in the same github repo)
Conduct the interview
Submit the full transcript with interviewer notes (your immediate impressions while they're fresh in your mind) via google form.

Open Questions:

[ ] Since we're all using our personal emails, can we use a CNCF alias or some other more official and standardized way of communicating with interviewees?

[ ] Is using google drive, forms, and sheets GDPR compliant when restricted to a small number of project leads?


Segment Anything CPP Wrapper for macOS

Thanks for your Segment Anything CPP Wrapper.
Here is the Segment Anything CPP Wrapper for macOS.
This code is originated from Segment Anything CPP Wrapper and implemented on macOS app RectLabel. We customized the original code so that getMask() uses the previous mask result called as low_res_logits and retain the previous mask array for undo/redo actions.
Please let us know your opinion.

Improved "Create polygon using SAM" feature so that you can label pixels using the pixels option. You can label Segment Anything 1 Billion (SA-1B) like dataset by yourself.

We respect your work and thanks for updating the script.
We updated Segment Anything CPP Wrapper for macOS so that it corresponded to MobileSAM, EdgeSAM, EfficientSAM, HQ-SAM, and Segment Anything models.


ESP32s3 heap capability allocator leads to memory fragmentation. (IDFGH-12587)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report
Please rethink the capability heap's allocation strategy.
Currently malloc appears to allocate from the largest heap first. This led to a fragmentation related allocation fault. It would seem better to place smaller objects in smaller heaps first by default. I think the largest heap is just the piece of contiguous memory left after statically scoped variables are placed by the complier.
Details
Yesterday I ran out of heap memory. More precisely, I had about 100k free of MALLOC_CAP_8BIT capable memory, needed 36k but the largest free block was only 34k. I was sad.
It appears the ESP32s3 (and probably other devices):

has five heaps: 1 in PSRAM, 3 in internal RAM (two of which appear equivalent) and 1 in RTC fast RAM.
the internal RAM heaps were roughly 200k, 20k and 32k. The 200k and 20k are in SRAM1 while the 32k one is in SRAM2.
malloc fills up the 200k ram first. This is why I was unable to obtain a 36k block of RAM.

For reference, here's a summary of my memory disposition:

Questions

Is it possible, using heap_caps_malloc to place an object in heap 3 or 4 in the diagram above? I can't see any combination of MALLOC_CAP_* flags that would let me do this before heap 2 was too small for the allocation.
Is there any documentation on replacing the default heap strategy with a custom one? I haven't been able to find it yet, unfortunately.
Is it possible for heap 2 and heap 3 to be combined into a single heap? Its not clear to me why these show up as two separate heaps. I saw some kind of post-init cleanup/heap gathering in the source. Is it related to this?

Hi @PaulMartinsen,
thanks for reaching out to us concerning this problem. Let me first address your questions.

It should be possible to allocate from either heaap2/3 or heap4 based on the CAPS from the table you shared. But unfortunately choosing between heap 2 and heap 3 is not.
you can't currently replace the heap allocation strategy but you can create a new heap at runtime in e.g., heap2 and then allocate in your newly created heap region. For that you can use heap_caps_add_region_with_caps() in esp_heap_caps_init.h. That can be a workaround that gives you the choice of where to allocate.
Upon startup a part of the memory has to be reserved because it is used as a stack. After the system has started, this reserved area is no longer needed and can be then used as heap. The currently implemented solution is to create heaps where possible upon startup and leave out those reserved area. When those areas are no longer needed as stacks, the heap component creates a new heap in place of those reserved areas. That is why you can observe that the RAM memory region is split into 2 different heap even though it could technically be covered by 1 heap only.

I am currently working on an alternative approach to the startup stack problem that will result in creating a single heap per memory region. That will solve the issue you are facing at the moment.

Thanks @SoucheSouche for your quick and helpful comments. Particularly for clarifying where heap 3 comes from. Your new approach to join heaps 2 & 3 sounds great and a lot better than allocating into heap 3 explicitly (which would be gone with the new approach). I think this will be a big step forward for us.
When I enabled support for malloc to use PSRAM (possibly a temporary work-around), I noticed there's an option to reserve a section of internal ram that can be used by the capability allocator and is not used by malloc. This does go into heap 2/3. I don't think we will need this, but I did wonder if having an option to place this in "heap 4" would be straight forward and worthwhile.
Thanks again for your comments and work on the memory management.

@PaulMartinsen, no problem. we can leave this ticket open and I will link it to the merge request for this heap initialization update so it gets closed when the new implementation gets merged in master.
The modifications are quite big so it might take a little while before I can merge.

Also seeing similar issues - unable to start camera because it wants to make a ~30kb dma-capable allocation, but largest single block of dram is only ~28kb, even though >125kb free dram.
esp32 classic does not have this problem afaics because the smaller heaps are first, so get used first, leaving most of the final (largest) heap free.
esp32s3 has the largest heap first, so that gets (partially) used, leaving part of it, and the smaller heaps, free - but no single large block available.

@SoucheSouche How is the status of this fix? https://github.com/espressif/tlsf/pull/2

@SoucheSouche How is the status of this fix? espressif/tlsf#2

@SoucheSouche When will esp-idf update tlsf submodule to get above fix? Will you update tlsf for release branches as well?

@AxelLin, the MR to update the tlsf submodule is created and under review now. No backport can be done for https://github.com/espressif/tlsf/pull/2 because backporting implies patching the ROM version of the TLSF and since this PR updates static inline functions, it simply can't be done.
@PaulMartinsen the new implementation of the heap init process is stalled because of ROM TLSF backport issue as well. In the mean time, I will probably end up sorting the heaps by increasing size to fix your issue and backport it.

@PaulMartinsen, I created a fix that sorts the heap by increasing size. It is currently under review.

@AxelLin, the MR to update the tlsf submodule is created and under review now. No backport can be done for espressif/tlsf#2 because backporting implies patching the ROM version of the TLSF and since this PR updates static inline functions, it simply can't be done.

@SoucheSouche
I don't quite understand this.
I notice the v5.4 branch includes the fix espressif/tlsf#2 .
Now I'm using v5.2 branch.
If rebuild my application using v5.4 branch and OTA firmware on the device, will it work?

@PaulMartinsen, I created a fix that sorts the heap by increasing size. It is currently under review.

@SoucheSouche How is the status now? Could you share the commit for this fix?

@AxelLin, the fix has been reviewed and will be merged soon. The commit will be made available when the work is merged and synced with GH.

@AxelLin, the fix has been reviewed and will be merged soon. The commit will be made available when the work is merged and synced with GH.

I won't call it soon if it takes more than 3 weeks.
You mentined you have created a fix 2 Months ago (https://github.com/espressif/esp-idf/issues/13588#issuecomment-2367703032), but the fix/review is so slow.
And it will take even more time for backping to stable branches.


Import single/Specific collections from Mongo

Hi,
I was trying to import data from my Mongo db to Elastiseach. It was loading data but took all collections. I need to load very specific set of collections say 2-3 out of 250 collections from mongo. How can i do that?
I am using latest version of abc.
Checks

[x] I have mentioned my abc version in the issue, if applicable.
[x] This issue is not a duplicate.
[x] This issue has a description.

My  Mistake, havent reviewed documentation thoroughly. Got it as how i can do same.


fix(test): provide more oomph for compat-test

Description
Switch to e2-standard-8 x 2 for this test (was e2-standard-4 x 3) to avoid unscheduled collectors that have appeared since GKE switched v1.26->1.27.

https://search.ci.openshift.org/chart?search=Timed+out+after+10m&maxAge=336h&context=1&type=build-log&name=stackrox.*compat.*&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
https://issues.redhat.com/browse/ROX-19023

Checklist

[ ] Investigated and inspected CI test results

Testing Performed
/test gke-version-compatibility-tests

version-compat passes, the other failures are known flakes.


[fbrp] direct command invokation

Description
Allow fsetup.py to invoke commands directly, outside the command line args.
For example:
import fbrp

...

fbrp.cmd.up(procs=["proc"], build=False, reset_logs=True)
fbrp.cmd.wait(procs=["proc"])
fbrp.cmd.logs(procs=["proc"], old=True)

Type of change
Please check the options that are relevant.

[ ] Bug fix (non-breaking change that fixes an issue)
[ ] Proposes a change (non-breaking change that isn't necessarily a bug)
[x] Refactor
[x] New feature (non-breaking change that adds a new functionality)
[ ] Breaking change (fix or feature that would break some existing functionality downstream)
[ ] This is a unit test
[ ] Documentation only change
[ ] Datasets Release
[ ] Models Release

Type of requested review

[ ] I want a thorough review of the implementation.
[x] I want a high level review.
[ ] I want a deep design review.

Before and After
...
Testing
Manual testing.
Checklist:

[x] I have performed manual end-to-end testing of the feature in my environment.
[ ] I have added Docstrings and comments to the code.
[ ] I have made changes to existing documentation where needed.
[ ] I have added tests that show that the PR is functional.
[ ] New and existing unit tests pass locally with my changes.
[x] I have added relevant collaborators to review the PR before merge.
[ ] [Polymetis only] I ran on hardware (1) all scripts in tests/scripts, (2) asv benchmarks.

The code changes look good to me. I also tested out the direct invokations in my local fbrp_ifttt example and it runs as expected. We could do arbitrary boot-up sequences now with chaining these I'd imagine; cool!


feat: Add ability to configure propagation style per job/queue in Sidekiq instrumentation

Currently, propagation style can be configured for the whole app and it will be applied to all jobs. If propagation_style is set to child, then it's convenient to see a trace where the two "producer" and "consumer" spans are shown next to each other and there is a little (usually several ms) between them.
However, in some cases, we schedule jobs to be performed over a longer period, for example in several days. Then if  propagation_style is set to child then visually the trace will be unreadable because there will be a huge gap between the spans where nothing happens.
This PR adds a new propagation_style_per_job option that allows dynamically choose propagation style per job or queue.
Example:
c.use 'OpenTelemetry::Instrumentation::Sidekiq',
  propagation_style: :child,
  propagation_style_per_job: lambda { |job_class_str, _queue|
    :link if job_class_str == 'SomeJob'
  }

@arielvalentin hey, I've opened the PR with working implementation but wanted to first get a feedback before moving further with tests, etc. I tested the changes manually and they work well. Please let me know if anything should be adjusted. Thanks!

👋🏼 @kirillkaiumov thank you for your submission.
We require that you review and sign the EasyCLA before we accept any PRs. When you have a moment please do so and we can start iterating on reviews for this PR.

We discussed this a bit during the Ruby SIG this on 2023-08-22 and we did not come to a consensus on how or if we want to allow for per worker propagation configurations.
@robbkidd @no-reply @bolandoboy @pollosp have been looking at the updates to from the messaging SIG and will be working on aligning the instrumentations to ensure they are spec complaint. Although I think it overlaps with the use cases described in the messaging spec, it does not prescribe how to configure automatic instrumentations or the granularity of instrumentation config.
That being said I will defer to the folks I mentioned above to get this input on a design direction here.
Please stay tuned

@robbkidd @no-reply @bolandoboy @pollosp hey! It's been 3 weeks already. Any feedback on the proposal that you can already provide?


[Roadmap] Cleanup for filters

After this filter release this week we need to do a slight iteration to fix some quirks with it.

fix 99% spread -> 100% all spreads thing
add unit tests
no longer assume there is infinite liquidity at the edges
Use dynamic text in market creation specific to a given user's market + show ahead of time whether their market will show up with default filters
show warnings on market pages that fail invalid filter
tooltip on filters on market page that has similar language as market creation one
Don't uncheck invalid markets box if someone owns shares of that market. What it should do is keep it filtered and if you go to the portfolio page you can see those markets because they’re in your portfolio
only show tags and categories based on what markets pass the filter. Rank tags and categories by liquidityTokens of markets within them
There’s also the drop down arrow bug where if you click on the filter dropdown arrow it doesn’t actually show the filters

✅ account for TVOM in PR today

Paul asked me to list these new items based on discussion with him, Joey and Chwy

make sellIncrement smaller so that markets with less liquidity survive spread filter. (Currently sell increment is 0.05 ETH / $12 USD, proposed 0.02 ETH / $4 USD)
liquidity filter also available as a standalone filter, eg. to sort by OI + liquidity filter
default homepage caps at 2% fee instead of 5% [requested by chwy, not yet approved by Joey]
gas prices are currently baked in when order book changes, we might want to apply them in getter so that filtered markets actually use current gas price

Not new, but note on UI+data design already being done:

"Use dynamic text in market creation specific to a given user's market + show ahead of time whether their market will show up with default filters" --> Cillian, Chwy and Ryan have completed a v2 mockup + data design for this, some of which we might want to backport into v1, and the backend data work applies to both v1 and v2

Infinite liquidity change is under debate at the moment
@ryanberckmans did you see this change + convo in conf-room?
"It’s really just take the invalid filter revenue minus cost idea, but change revenue to be revenue if the market resolved at the max for the bid and the min for the ask. Only count liquidity in the liquidity score where revenue - cost > 0"
Basically don't count orders where it'd be unprofitable to take them even if the market resolved at the extremes

@joeykrug saw the change + convo in conf-room. Change makes sense!

I believe this is done with the last commit.

Re-opening, none of the subtasks for account for...

@bthaile
i think this one got done earlier, am I wrong?:

There’s also the drop down arrow bug where if you click on the filter dropdown arrow it doesn’t actually show the filters

These UI tasks are left (These should be a part of #2366)

Use dynamic text in market creation specific to a given user's market + show ahead of time whether their market will show up with default filters
Show warnings on market pages that fail invalid filter
tooltip on filters on market page that has similar language as market creation one
Don't uncheck invalid markets box if someone owns shares of that market. What it should do is keep it filtered and if you go to the portfolio page you can see those markets because they’re in your portfolio


Added Azure.Firewall.PolicyMode

PR Summary
Added Azure.Firewall.PolicyMode rule.
Fixes #2354
PR Checklist

[x] PR has a meaningful title
[x] Summarized changes
[x] Change is not breaking
[x] This PR is ready to merge and is not Work in Progress
Rule changes

[x] Unit tests created/ updated
[x] Rule documentation created/ updated
[x] Link to a filed issue
[x] Change log has been updated with change under unreleased section

Other code changes

[ ] Unit tests created/ updated
[ ] Link to a filed issue
[ ] Change log has been updated with change under unreleased section

@BernieWhite Ready for review, let me know what you're thinking.


Draw cubes, spheres within pv

Hello! Is there an established way to render a set of geometric primitives (e.g., cubes, spheres, etc.) within pv?
I want to drop a bunch of translucent cubes overlaid on top of a protein structure.

Hi,
there is basic support for adding custom geometry to the scene. This sample shows how to add spheres and tubes:
http://pv.readthedocs.org/en/v1.8.1/sample-custom-mesh.html
There is currently no support for rendering  cubes, but that can be added easily. How do you want the cubes to be aligned? Is it OK if they align to the worlds x, y and z axis or do you want to specify the corner points directly?
Best,
Marco

Hi Marco, thanks for pointing me to the example.
Yes, I just need cubes that are axis-aligned to the world's x.y and z. I'm intending to use the cubes to render a (coarse, ~1A) 3d mesh of voxels over a structure. I'd love to be able to control the alpha and color of the cubes as well. Thanks!

I see, I should have some time next week to implement it. Will get back to you after that.

@biasmv Could you provide some clues of implementing the cubes drawing ? Thanks very much

@biasmv Could you provide some clues of implementing the cubes drawing ? Thanks very much


Fix : Added resetMaterialTextField to FriendlyChat prototype function

Added resetMaterialTextField to FriendlyChat prototype function

Thanks for looking into this @arunabhdas, however this was done on purpose. Basically this method is static (it doesn't need to access any object attributes).
Thanks anyways!

It throws an exception in Javascript console when you run firebase serve though.

Could you paste the exact error and tell us what browser/version you are
using?
On Sat, 3 Jun 2017 at 13:43, Arunabh Das<EMAIL_ADDRESS>wrote:

It throws an exception in Javascript console when you run firebase serve
though.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/firebase/friendlychat/pull/218#issuecomment-305969825,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADl5hzd6msG4SlmED3ZVFfjQuTUkU8G7ks5sAUbQgaJpZM4NtzqE
.


Pretending D3D video memory capabilities

I got a case (here: https://sourceforge.net/p/dxwnd/discussion/general/thread/6d74dbdff0/?page=1#821c ) where a DirectShow movie rendering doesn't work unless we make some tricks. Someone noted that the rendering is good when the video adapter exposes some capabilities that are not present on the failed situations, in particular these:
D3DDEVCAPS_TEXTUREVIDEOMEMORY
D3DDEVCAPS_SEPARATETEXTUREMEMORIES
D3DDEVCAPS_PUREDEVICE
Now, assuming (but not for granted) that the problem is about the missing separated texture video memory, the question is: is there an easy way to try pretending to have these capabilities?
The implementation in the GetDeviceCaps wrapper is rather trivial, but once I pretend that these capabilities are present, how should I redirect a request for texture (separated) video memory to system memory and avoid a failure?
More in general, what do you think about the idea? Is it sound or are we just wasting our time?
Thanks.
p.s. the DirectShow rendering doesn't throw any error, it works but simply shows a light-blue screen and no errors are reaching the application interface level.

D3DDEVCAPS_TEXTUREVIDEOMEMORY

This should always be present for any 3D capable GPU, so I doubt any software even checks for it. Hardware that supports texturing directly from system memory probably never existed. Yes, integrated graphics may be technically using system RAM, but it's dedicated RAM that the drivers treat as video memory, so you still have to create textures in "video memory" (from DX point of view) as far as I know.

D3DDEVCAPS_SEPARATETEXTUREMEMORIES
D3DDEVCAPS_PUREDEVICE

It seems the meaning of the first flag changes depending on which docs and DX version you check, and the second one is not very descriptive in DX9, so I won't even try to guess what these are supposed to do. I never bothered with them.

More in general, what do you think about the idea? Is it sound or are we just wasting our time?

Sounds like a waste of time to me. That user may have one of those GPUs that use D3D9On12 unless they are using very recent drivers, so the problem may be with that. If he does have very recent drivers that uses igdumdim32.dll / igd9trinity32.dll instead, then that might be the culprit.
I doubt it has anything to do with those caps.


Different Compile Method's

Im a Windows 7 User, and i can't get pass in the "npm install", even if i used powershell, can you make a version compatible with MSYS2/MinGW?

just use anything compatible with node, windows 7 is kinda out of support and modern software may not work as well on it./

Nevermind i had to use Node 12


Shows a border gradient on android in release

This issue doesn't appear in simulator, it happens in a physical device.
Adding Root to Main function  or any screens causes this.

import { AppRegistry } from 'react-native';

import { name as appName } from './app.json';
import { App } from './src/app';
import { Root as PopupRootProvider } from 'react-native-popup-confirm-toast';

export default function Main() {
  return (
    <PopupRootProvider>
      <App />
    </PopupRootProvider>
  );
}

AppRegistry.registerComponent(appName, () => Main);

This function uses toast component, similarly Popup in other screens.
 const deleteMyAcc = (input: FormType) => {
    deleteAccount(
      {
        ...input,
      },
      {
        onSuccess: (s) => {
          Toast.show({
            type: 'success',
            title: 'Account Delete',
            textBody: 'Account deleted successfully',
            buttonText: 'OK',
            callback: () => Toast.hide(),
          });
          signOut();
          navigation.replace('Auth');
          console.log(s);
        },
        onError: (e) => {
          console.log(e.response?.data);
          showError(e);
        },
      }
    );
  };

@AngelKrak It also makes any buttons underneath the view unclickable

same issue here

@sekizlipenguen It also makes any buttons underneath the view unclickable

I have done fix for android phones before. When Toast is active, click is turned off in the whole view.
Do you have a chance to send a more detailed screenshot or screen recording?

If you have set the status bar property "translucent" to true for android, you may be experiencing this problem.

I hope this version will fix the problem.
https://www.npmjs.com/package/react-native-popup-confirm-toast
version: 2.3.2


pullapprove: remove it

We are moving off pullapprove. Remove its config file.
Fixes: #1215
Signed-off-by: Graham Whaley<EMAIL_ADDRESS>

/test

/test

We don't need CI results to pass here - we're simply removing a config file. Merging...

Webhook removed.


add new bets member to the group

Config changes proposed in this pull request

For new organization members
I acknowledge that

[ ] I have read and understood the Open Source Guidelines
[ ] I agree to act accordingly (with due dilligence) to our guidelines
[ ] I have no open questions regarding the topics covered (please delete the "open questions" section)

Open questions

Config changes proposed in this pull request

For new organization members
I acknowledge that

[ ] I have read and understood the Open Source Guidelines
[ ] I agree to act accordingly (with due dilligence) to our guidelines
[ ] I have no open questions regarding the topics covered (please delete the "open questions" section)

Open questions

Config changes proposed in this pull request

For new organization members
I acknowledge that

[ ] I have read and understood the Open Source Guidelines
[ ] I agree to act accordingly (with due dilligence) to our guidelines
[ ] I have no open questions regarding the topics covered (please delete the "open questions" section)

Open questions


[Modules] Default value for IpConfiguration name

Description
#1278
Added a default value for the Ip configuration of network interfaces. Now if no value is provided for the IP configuration, the IP configuration is named 'ipconfig-{currentIndex}'.

Whilst there are other IP configuration name values across the modules (e.g., Azure Firewall), only the Virtual Machine is updated. The rationale being the name for an AZ Firewall IP configuration is more significant than that within a network interface.
Pipeline references

Pipeline

Type of Change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

Checklist

[x] I'm sure there are no other open Pull Requests for the same update/change
[x] My corresponding pipelines / checks run clean and green without any errors or warnings
[x] My code follows the style guidelines of this project
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation (readme)
[x] I did format my code

@ahmadabdalla ready for round 2 when you are.

Thanks @ahmadabdalla! I do not have access to merge. I'll leave for you whenever it suits


Sitemap anchor issue

[ ] No discrepancies across browsers. Eg. chrome vs safari
[ ] Pages pass W3C Validation for HTML (https://validator.w3.org)
[ ] CSS (https://jigsaw.w3.org/css-validator/)
[ ] Google sign-in is operational
[ ] Accessibility functions
[ ] Pages can scale without distorting page
[ ] No dead links
[ ] Navbar is consistent across the site
[ ] Pages are screen-reader accessible
[ ] Contrast meets standards for accessibility
[ ] Pages are keyboard accessible
[ ] Passes WAVE Evaluation Tool chrome extension

@dm9965 great work with this i had no clue you could use ids for anchors


Determine flashable memory regions based on flash algorithms

Previously, we used the memory_map entries in the target description
file to determine whether an NVM address was flashable or not. However,
some target CMSIS-Packs did not include ranges for all flashable areas
(e.g. EEPROM), even though a corresponding flash algorithm exists.
However, if we have a flash algorithm, then we can flash this memory
region, even if no corresponding entry in the memory_map exists.
This commit updates the flash loader to use only the flash algorithms to
determine whether an NVM address is flashable or not.
Based on #482, merge that first!

Hm, this now conflicts with #480.
I'm not sure if the flash API is the proper location for writing to RAM. I haven't taken a close look at it, but it feels like writing to RAM should be a separate function, instead of hacking it into the flashing API...

Hm, this now conflicts with #480.
I'm not sure if the flash API is the proper location for writing to RAM. I haven't taken a close look at it, but it feels like writing to RAM should be a separate function, instead of hacking it into the flashing API...

Well, writing to RAM is a separate function. The flash API was chosen to also work because then, people can use cargo-flash/embed/probe-run to write a program directly to flash if they want/need/like to.

There's one problem with this approach: Usually the flash algo specifies the "biggest" possible address range, but the actual chip may have less flash.
For example, the nrf52833 flash is 0x0 - 0x80000, but the flash algo says it can flash 0x0 - 0x200000. We want flashing to addresses > 0x80000 on the nrf52833 to fail, but with these code changes it'll try to do it anyway.

@Dirbaio damn, you may be right... just as I was polishing the last details of my PR :cry:
However, how would you "fix the target YAMLs"? I'd assume that the same issue applies to EEPROM flash algo ranges as well. In that case, wed have no way to know about the actual EEPROM address range from the CMSIS packs alone. We'd need to consider additional data sources, which will probably be different for every manufacturer.

yeah, it'd have to be manual :(
like in #500

@Yatekii @Tiwalun what do you think about this? Note that we'd also need some kind of tooling to merge the manually updated files with the re-generated files from the CMSIS packs.

Hey @dbrgn sorry, I completely forgot about this ... I think @Dirbaio makes a good point here and we should actually do this in the Pack files even tho this means a lot of work goes to waste :/
Concerning patching: Yes, this is an issue. I am not sure how to do this best, but maybe we could store a list of diff patches that we apply on the .pdsc file before we parse it? So changes would be honored.
What do you think?

Hey @dbrgn sorry, I completely forgot about this ... I think @Dirbaio makes a good point here and we should actually do this in the Pack files even tho this means a lot of work goes to waste :/
Concerning patching: Yes, this is an issue. I am not sure how to do this best, but maybe we could store a list of diff patches that we apply on the .pdsc file before we parse it? So changes would be honored.
What do you think?

How often are the packs updated? What other tools (pyocd, openocd) seem to be doing is to simply maintain their own target databases, not "updating" them from the packs.
We also already have "patches" in the nrf yamls without any diffs recording them.

How often are the packs updated? What other tools (pyocd, openocd) seem to be doing is to simply maintain their own target databases, not "updating" them from the packs.
We also already have "patches" in the nrf yamls without any diffs recording them.

I am not sure how often they are updated. I think we only had to update two packs form manufacturer updates. I think all other changes were done manually.
I am aware that there is already diverging target descriptions.
Regarding maintaining our own source of truth: That's actually one of three reasons I wanted the yaml files and not just the pack files, so we can patch things easily. The other reasons being having a universal format also supporting other architectures and getting rid of all the pack bloat and garbage.

I am not sure how often they are updated. I think we only had to update two packs form manufacturer updates. I think all other changes were done manually.
I am aware that there is already diverging target descriptions.
Regarding maintaining our own source of truth: That's actually one of three reasons I wanted the yaml files and not just the pack files, so we can patch things easily. The other reasons being having a universal format also supporting other architectures and getting rid of all the pack bloat and garbage.

@Yatekii should we close this?

Hmm I am not sure anymore, but if I remember correctly, we should not do what this PR does.
Or do you have any other strong oppinion?


Dupe Bug with silk touch

if you break a rtftools machine with silk touch on the pick (tinkers and vanilla) you get a duplicate of the machines. This is on rftools 2.71.

Perhaps fixed next release.


The minimum supported version.

I adjusted the version and found that UIViewPropertyAnimator requires iOS10 or higher. Why not support iOS10? Is there something I didn't find?

Duplicate issue, the reason why we don't support iOS 10 or lower has been listed in #62


[Feature] Rancher 2.0 & Cluster Autoscaling

Dear Rancher,
is there any plan to support for this feature which is supported since 1.10 on Azure for instance.
https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider
https://forums.rancher.com/t/node-autoscaling/9141
Thanks & Regards

https://github.com/jepp2078/autoscaler/tree/rancher-cloud-provider
https://github.com/kubernetes/autoscaler/compare/master...jepp2078:rancher-cloud-provider
@jepp2078 did some work on this 🙌

Is there any update on when this might be added to Rancher?

@casz  Do you know if the rancher provider works with v2 rancher?

@tmedford it's for v2.

I also looked at the State of the Cluster Autoscaler, and was clearly disappointed! I am talking about it in general and not about @jepp2078 addition to it, which is going in the right direction, but also needs some more work. I am writing down my thoughts about this here after the paragraph about the autoscaler itself.
So for everyone else, looking into this: The cluster autoscaler is if you're thinking like me, not what you think it is. It is not by far taking any action to make new machines available, it is (at least yet) a dump stupid unit that increases and decreases numbers based on resource requests. So throwing in  the autoscaler only works, if you have an auto scaling set in place that is capable of setting itself up to the point of automatically joining the cluster. The cluster autoscaler does not instruct, it only adjusts the number of needed hosts in the autoscaling set.
So for rancher: The implementation of @jepp2078 looks very good already. But it lacks support for scaling to zero, which is more a problem of rancher currently, not allowing this to be done. And the second thing is, to make the autoscaler work properly with rancher, I think it needs to be supported by rancher as a concept in the first place.
So here are my thoughts how it could look like:
Rancher will need a node pool that can be configured just like an auto scaling set. Pre defining the maximum amount of hosts possible in it, the minimum and the capability to scale down to zero and additionally be able to pass an arbitrary json to it. The api needs to return the following information about the node pool:

in which cloudprovider the host is
the instance (name) type selected
the scaling information (min, max, demanded, current)
the arbitrary json

The autoscaler will need to interpret this api information as the following: First take the cloudprovider and if avalable logic and templates into account. For example for aws take https://github.com/kubernetes/autoscaler/blob/c756ed3953dfd3abc43894e2034964b0853d438f/cluster-autoscaler/cloudprovider/aws/ec2_instance_types/gen.go and if available also the pricing information https://github.com/kubernetes/autoscaler/blob/c756ed3953dfd3abc43894e2034964b0853d438f/cluster-autoscaler/cloudprovider/gce/gce_price_model.go like for gce.
If it is a non supported provider, then from the arbitrary json information about the instance can be provided by the user (like instance type is gpu or not, also pricing information, whatever the user needs here).
There is certainly room for improvement on that, but with that in place we would have a pretty good stance for the autoscaler, which is in my view even superior to all the current ones. Since those are strictly bound to the cloud and have been exclusively build for them (which is pretty much a reflection of the development of k8s in general..., it is often petty much bound to the environment instead of being decoupled and being usable everywhere).
So I think this shouldn't be too hard to implement. Since the node pool already somehow has support for scaling to zero. At least you can write a zero manually into the form and it will actually be saved. So it would need to be extended with the bits mentioned.
I am not sure if I will find the time to look into that into my self, since this takes considerable amount of time, as I have not dived through ranchers core source yet much. But it is generally a topic I would like to see happen very soon, since I have an active need for it. So I will look how I can support to help make it happen.

Would it be possible to have a separate nodepool which acts as nodepool only for autoscaling?
We are using rancher with the official terraform provider where we define our nodetemplates and define a specific number of host to each of them.
The autoscaling machansim should not interfere with that state otherwise the state will drift.
When autoscaling happens and we want also to apply some changes via terraform it would be bad that terraform overwrites the current node number on the nodepool.

We are launching K8S clusters using Terraform and the Rancher2 provider.
Worker nodes are always part of an Auto Scaling Group (ASG), each ASG is tagged accordingly so that cluster-autoscaler can detect them and scale up/down:

KubernetesCluster - <cluster_name>
k8s.io/cluster-autoscaler/enabled - true
k8s.io/cluster-autoscaler/<cluster_name> - <cluster_name>

Each worker node registers with rancher using docker command in user-data:
docker run \
    -d \
    --privileged \
    --restart=unless-stopped \
    --net=host \
    -v /etc/kubernetes:/etc/kubernetes \
    -v /var/run:/var/run \
    rancher/rancher-agent:v2.2.5 \
        --server https://rancher-rnd.example.com \
        --token <token>  \
        --worker

This allows me to scale up workers freely, it takes a couple of minutes for a worker to be ready from the time it is launched in EC2 until it starts all the required components.
This is the easy part, it gets hard when you try to scale down, since terminated workers still remain in rancher in status Kubelet stopped posting node status..
My solution:
Create user in rancher with name node_remover, give it access to clusters you want to autoscale with Manage Nodes role, create an API key and save it somewhere.
To remove workers from rancher when scaling down I use a script that runs on instance termination:
#!/bin/bash
# chkconfig: 0123456 99 01
# description: EC2 Termination script
. /etc/rc.d/init.d/functions
start() {
    /usr/bin/touch /var/lock/subsys/ec2-termination
}
stop() {
    node_ip=`/usr/bin/curl -s http://<IP_ADDRESS>/latest/meta-data/local-ipv4`
    node_id=`/usr/bin/curl -k https://rancher.example.com/v3/nodes/?clusterId=<cluster_id> \
      -su "<user>:<pass>" \
      -H 'Accept: application/json' \
      | /usr/bin/jq -r --arg ip_address "${node_ip}" \
      '.data | .[] | select(.ipAddress == $ip_address) | .id'`
    /usr/bin/curl -k -XDELETE "https://rancher.example.com/v3/nodes/${node_id}" \
      -u "<user>:<pass>" \
      -H 'Accept: application/json'
    /usr/bin/rm -f /var/lock/subsys/ec2-termination
}
case "$1" in
    start)
       start
       ;;
    stop)
       stop
       ;;
    restart)
       stop
       start
       ;;
    status)
       ;;
    *)
       echo "Usage: $0 {start|stop|status|restart}"
esac
exit 0

You need to know cluster_id in advance, this is east with Terraform since it created the cluster in rancher so I just pass the output to user-data.
The final user-data looks like this:
#cloud-config

packages:
  - ca-certificates
  - docker
  - jq

write_files:
  - path: /etc/init.d/ec2-termination
    owner: root:root
    permissions: '0755'
    content: |
      #!/bin/bash
      # chkconfig: 0123456 99 01
      # description: EC2 Termination script
      . /etc/rc.d/init.d/functions
      start() {
          /usr/bin/touch /var/lock/subsys/ec2-termination
      }
      stop() {
          node_ip=`/usr/bin/curl -s http://<IP_ADDRESS>/latest/meta-data/local-ipv4`
          node_id=`/usr/bin/curl -k https://rancher.example.com/v3/nodes/?clusterId=<cluster_id> \
            -su "<user>:<pass>" \
            -H 'Accept: application/json' \
            | /usr/bin/jq -r --arg ip_address "${node_ip}" \
            '.data | .[] | select(.ipAddress == $ip_address) | .id'`
          /usr/bin/curl -k -XDELETE "https://rancher.example.com/v3/nodes/${node_id}" \
            -u "<user>:<pass>" \
            -H 'Accept: application/json'
          /usr/bin/rm -f /var/lock/subsys/ec2-termination
      }
      case "$1" in
          start)
             start
             ;;
          stop)
             stop
             ;;
          restart)
             stop
             start
             ;;
          status)
             ;;
          *)
             echo "Usage: $0 {start|stop|status|restart}"
      esac
      exit 0

runcmd:
  - chkconfig ec2-termination --level 12345 on
  - chkconfig ec2-termination --level 06 off
  - service ec2-termination start
  - systemctl start docker
  - systemctl enable docker
  - usermod -a -G docker ec2-user
  - docker run -d --privileged --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.2.5 --server https://rancher.example.com --token <token>  --worker

I am working on something similar but more specific to GPU nodes. But the overall approach is the same:

custom rancher cluster
cloud-config to register with rancher and install any custom tools

the difference that I am building a custom auto-scaler for now before something is ready from the Rancher team.

I don't see a reason to build a custom autoscaler when Kubernetes provides its own official autoscaler.
Also Nvidia has Kubernetes support, so I believe cluster autoscaler can scale based on gpu metrics too.

@azelezni I am not suggesting anyone doing it, it's specific to our situation. As far as I know, official autoscaler doesn't allow you to have cloud-init for your nodes, if it did - would be awesome. If you check support for Azure specifically, it goes only for clusters built with aks-engine or managed azure service and we are using Rancher.

And one more reason doing it now is because Rancher will probably have to consider N variables/platforms before offering cluster-autoscaler, we just need one on Azure for now, so complexity on our side is much smaller than one of a generic solution.

The autoscaler is probably the most wrongly understood and overrated component. It is currently only designed to work with clouds and is yet actually quite limited in its functionality. When rancher matures its node pool concept, actually they would get a huge advantage in front of the whole market, b/c combined with an adaption for the rancher api on the autoscaler, this would mean, that the autoscaler for the first time would be platform agnostic, instead of relying on autoscaling sets on the side of the provider and an actual implementation behind it.
To the support argument, I disagree, there is not much support needed for a first version of this. First of all you can reuse and reflect the logic in the autoscaler that exists for aws, gce and azure to retain the functionality for the scaling strategy. However for all other platforms all you would need is to offer an option, to define a predefined json structure for your node pool, which contains information about the pricing, the specs and other related things. Everything else is already perfectly handled by the current docker-machine provisioning setup. If you want ultimate flexibility, allow the user to pass an url to a service that the user can create, from which they can retrieve the information dynamically, where the user might just write a passthrough for the specific cloud.

To the support argument, I disagree, there is not much support needed for a first version of this. fingers crossed :) I will gladly switch to it when it's done - having auto-scaler working as part of the cluster with node pools is much better user experience than custom cluster where you can't define node pools (or maybe you can, but I haven't found documentation on that)

I already did some work on this at my last employer @wzrdtales.
This fork of the official autoscaler, has a functional implementation of a cloud provider for the rancher api.
https://github.com/jepp2078/autoscaler/tree/rancher-cloud-provider
Back then some work was required from ranchers side, since scaledown was handled differently in rancher versus the autoscaler (rancher scales down by removing the newest node, autoscaler does so by removing the most under-used node).

yup ik @jepp2078 and I already evaluated it. It fell through though, since it is too limited yet. But that is due to ranchers unmatured node pool concept that doesn't fit this use case currently. I also fall back for now to a manual cluster that places in the cloud init config the rancher boot up. The only advantage of that approach is that it works on multiple clouds/datacenters with the same cluster.

This is the easy part, it gets hard when you try to scale down, since terminated workers still remain in rancher in status Kubelet stopped posting node status.

This should be fixed in 2.3. Rancher will respect kubectl get nodes as the state of the cluster and the UI will reflect that.

Hi, Are we already able with 2.3.1 to perform cluster autoscaling on openstack driver?

Hi, Are we already able with 2.3.x to perform cluster autoscaling on vsphere driver?

This seems wrong - the cloud plugins / controllers are supposed to trigger removal of node on scale down.

So progress is: We have taints and removal support. So we're only lacking a custom json field on the node pool to store configuration on and we're ready to go to to adjust the cluster autoscaler component accordingly.

Presumably this will require a slight api change- take node pools from "number of nodes" to "minimum/desired/maximum" like an AWS asg?
Have to admit, was pretty suprised when I realized this didn't already work- figured that was the whole reason for the node pool concept in the first place.

Is there an ETA on when this feature will be implemented in rancher? We are currently evaluating rancher to potentially switch all our clusters to it but it fell short on autoscaling which is necessary for many of our cluster. We can delay switching to rancher if this is going to be released some time soon. Thanks!

Is there any progress of this feature request?
I think autoscaling is an important part of the puzzle that is missing in Rancher.
Most of production environments use this feature, not need to explain why.
it will lift up Rancher usage in the community.

Any progress on this?
I'm currently evaluating this and cluster autoscaling is a hard requirement for production usage.
Can you maybe share some new info @maggieliu ?
It would be highly appreciated.

My company is also looking for an easy solution for auto scaling nodes from within rancher.

I write my custom microservice for rancher api, https://github.com/Negashev/rancher-scale-webhook (docker negash/rancher-scale-webhook)
by CPU reservation and Rancher Alerts by Prometheus metrics with cordon scale down (sorry without documentation)
  containers:
  - env:
    - name: RANCHER_CORDONED_CPU
      value: "7" <= reserved 7% cpu (first hook corden and next hook down)
    - name: RANCHER_NODEPOOL_URL
      value: https://rancher.server.com/v3/nodePools/c-dddas:np-v5dud
    - name: RANCHER_TOKEN
      value: token-s5677:w45ve45e45v
    - name: TOKEN
      value: my-tokec
    image: negash/rancher-scale-webhook

logs of scale service

Alerts in Cluster (with timing , check to uncordon/scale up every 3 minutes, check to cordon/scale down  every 10 minutes)

Alert with kube_pod_container_resource_requests, kube_node_status_capacity_cpu_cores and kube_node_spec_unschedulable

Just use Cluster Autoscaler, it works great and with the new versions of rancher there is no issue of missing nodes.

Just use Cluster Autoscaler, it works great and with the new versions of rancher there is no issue of missing nodes.

But why is rancher than not listed as a supported deployment method in the readme?
https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/README.md#deployment

Because Rancher is not a cloud provider, Cluster Autoscaler works directly with the cloud provider your cluster runs on (AWS, GKE, etc...), Rancher too uses the cloud provider to provision the cluster.

Okay, sorry, that may be a mixup of terminology on my side.
the point is, however, that the cluster autoscaler project has no direct support to run on openstack, as far as I can see? there seems to be some support for magnum, will investigate this further.
Thanks for the information so far!

At this time, the Kubernetes cluster autoscaler works with the cloud providers on Rancher. Rancher 2.3 improved node reconciliation that addressed nodes being left in an orphaned state when scaling down. Support was also added for the horizontal pod autoscaler in the UI along with Prometheus metrics integration. These capabilities should allow users to autoscale on the major community-supported IaaS layers.
Autoscaling Rancher node pools is on our roadmap and we will provide updates here when we have more info to share.

@cloudnautique Is there any documentation regarding setting up cluster autoscaling?

At this time, the Kubernetes cluster autoscaler works with the cloud providers on Rancher. Rancher 2.3 improved node reconciliation that addressed nodes being left in an orphaned state when scaling down. Support was also added for the horizontal pod autoscaler in the UI along with Prometheus metrics integration. These capabilities should allow users to autoscale on the major community-supported IaaS layers.
Autoscaling Rancher node pools is on our roadmap and we will provide updates here when we have more info to share.
@cloudnautique
Would you have any idea of an eta or version that we should expect this feature? If not, would you be able to share any details of how this is best approached from a Rancher point of view as a workaround until then? This would be very interesting! Many thanks!

i made this,
https://github.com/guyta-essence/rna
it can help

Trying to avoid cloud provider route(GKE,AKS,EKS) and go bare metal using cloud infra.  Get more control that way. But, need cluster auto scaler to work at a minimum to at least enable us to add nodes if a new cluster is requested which requires more nodes.  Could this work if we used auto scale groups (AWS), or VM Scale Sets (Azure), as the VM type within Rancher node pool?  Would this enable us to achieve auto scale?

Trying to avoid cloud provider route(GKE,AKS,EKS) and go bare metal using cloud infra. Get more control that way. But, need cluster auto scaler to work at a minimum to at least enable us to add nodes if a new cluster is requested which requires more nodes. Could this work if we used auto scale groups (AWS), or VM Scale Sets (Azure), as the VM type within Rancher node pool? Would this enable us to achieve auto scale?

have a look at my solution https://github.com/guyta-essence/rna

Thanks Guy.  Looks like a good workaround.  It looks like it triggers based
on resource utilization metrics.  Can it also trigger based on a request
from the Kubernetes Auto scaler because a new pod is being created which
needs more CPU?  Also, how do you integrate this within Rancher?
On Tue, Jun 30, 2020 at 1:33 AM Guy Tadmor<EMAIL_ADDRESS>wrote:

Trying to avoid cloud provider route(GKE,AKS,EKS) and go bare metal using
cloud infra. Get more control that way. But, need cluster auto scaler to
work at a minimum to at least enable us to add nodes if a new cluster is
requested which requires more nodes. Could this work if we used auto scale
groups (AWS), or VM Scale Sets (Azure), as the VM type within Rancher node
pool? Would this enable us to achieve auto scale?
have a look at my solution https://github.com/guyta-essence/rna
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/rancher/rancher/issues/15145#issuecomment-651643847,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO75HOWKJ23QUE6HIWFBTKDRZGPODANCNFSM4FQQJHIQ
.

@Negashev

I write my custom microservice for rancher api, https://github.com/Negashev/rancher-scale-webhook (docker negash/rancher-scale-webhook)
by CPU reservation and Rancher Alerts by Prometheus metrics with cordon scale down (sorry without documentation)
  containers:
  - env:
    - name: RANCHER_CORDONED_CPU
      value: "7" <= reserved 7% cpu (first hook corden and next hook down)
    - name: RANCHER_NODEPOOL_URL
      value: https://rancher.server.com/v3/nodePools/c-dddas:np-v5dud
    - name: RANCHER_TOKEN
      value: token-s5677:w45ve45e45v
    - name: TOKEN
      value: my-tokec
    image: negash/rancher-scale-webhook

logs of scale service

Alerts in Cluster (with timing , check to uncordon/scale up every 3 minutes, check to cordon/scale down every 10 minutes)

Alert with kube_pod_container_resource_requests, kube_node_status_capacity_cpu_cores and kube_node_spec_unschedulable

Would it be possible for you to provide additional information for setting up in Rancher? This seems like a straight forward solution for use with node pools from node drivers that have the number of nodes in that pool accessible to change from API.

@jonathon2nd
  containers:
  - name: scale
    env:
    - name: RANCHER_CORDONED_CPU
      value: "7" # if <= reserved 7% cpu "first hook corden and next hook down"
    - name: RANCHER_NODEPOOL_URL
      value: https://rancher.server.com/v3/nodePools/c-dddas:np-v5dud # pool url (from api) for scale
    - name: RANCHER_TOKEN
      value: token-s5677:w45ve45e45v
    - name: TOKEN
      value: redacted
    image: negash/rancher-scale-webhook

service for scale with example IP <IP_ADDRESS>
post webhook scale up
curl -XPOST http://<IP_ADDRESS>:8080/up/redacted
post  webhook scale down
curl -XPOST http://<IP_ADDRESS>:8080/down/redacted
when you deploy webhook service to rancher (with rancher monitoring and two notify with UP and DOWN webhooks configs) you need to config alerts (UP and DOWN) by Prometheus query:
ALERT for my gitlab ci VMs with 8 cpu
UP Greater Than 0.7:
(sum(kube_pod_container_resource_requests{namespace="gitlab-managed-apps",node=~"git-ci-worker.*",resource="cpu"}) or vector(2)) / sum(kube_node_status_capacity_cpu_cores{node=~"git-ci-worker.*"} - (kube_node_spec_unschedulable{node=~"git-ci-worker.*"} * 8))

DOWN Greater Than 2:
(sum(kube_node_status_capacity_cpu_cores{node=~"git-ci-worker.*"})/(sum(kube_pod_container_resource_requests{resource="cpu",namespace="gitlab-managed-apps",node=~"git-ci-worker.*"}) OR vector(8))) OR vector(0)

@Negashev I was able to get this working, thanks!

NOT PRODUCTION READY
Thanks for @luthermonson
By this video
https://info.rancher.com/kubernetes-master-class-apr6-2021
I can install https://github.com/kubernetes/autoscaler

You need in cloud provider on our k8s cluster for node(s).spec.ProviderID - it's important for kubernetes/autoscaler
Add catalog https://github.com/kubernetes/autoscaler to rancher
Add secret to cluster-autoscaler-cloud-config to kube-system

apiVersion: v1
kind: Secret
metadata:
  name: cluster-autoscaler-cloud-config
  namespace: kube-system
type: Opaque
stringData:
  cloud-config: |
    [Global]
    url=https://rancher.com/v3
    access=token-tttttt
    secret=secretsecretsecretsecretsecret
    cluster-id=c-qwerty

Deploy autoscaler app v 9.9.2 from catalog (from step 2) to kube-system (yes you can add pools to autoscalingGroups[])

autoscalingGroups[0].maxSize: 10
autoscalingGroups[0].minSize: 1
autoscalingGroups[0].name: c-qwerty:np-qwerty
cloudProvider: rancher
extraArgs.cloud-config: /config/cloud-config
extraVolumeSecrets.cluster-autoscaler-cloud-config.mountPath: /config
extraVolumeSecrets.cluster-autoscaler-cloud-config.name: cluster-autoscaler-cloud-config
image.repository: luthermonson/autoscaler
image.tag: rancher

Try to help on https://github.com/kubernetes/autoscaler/pull/4041

@Negashev  What does the following lines corresponds to when we are using rancher as the cloud provider
extraArgs.cloud-config: /config/cloud-config extraVolumeSecrets.cluster-autoscaler-cloud-config.mountPath: /config extraVolumeSecrets.cluster-autoscaler-cloud-config.name: cluster-autoscaler-cloud-config

For rke2 (work on rancher 2.6.4)
https://github.com/shpwrck/rancher-clusterapi-autoscaling

https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider/rancher