Freeze vectors at the same end LSN

Looks correct at a quick glance. Needs more comments.
Do we need the new last_lsn value? It seems unnecessary to me.

Looks correct at a quick glance. Needs more comments.
Do we need the new last_lsn value? It seems unnecessary to me.

How else we can efficiently calculate end_lsn of open layer?
Originally we freeze open layers and last_record_lsn. But in this case created layers will always intersect with remaining open layers. So I want to calculate exact upper bound for end_lsn of evicted layers.

last_record_lsn works, no? What's the difference?

last_record_lsn works, no? What's the difference?

Oh, now I think i get it. An open layer might contain more recent WAL that we are not going to flush to disk in this checkpoint. So iteratively, if we flush layer A that contained a record with LSN X, we must also flush any other layers that contain a record with LSN <= X, too. That's what the 'freeze_end_lsn' tracks.
If the workload consists of random updates, it's likely that we will end up flushing everything on every checkpoint, because every in-memory layer will contain records close the last_record_lsn.
Perhaps we should just always flush everything that's in memory. Would seem simpler. So the logic would be that if there are any in-memory layers with data older than checkpoint_distance, we flush everything to disk. Otherwise, we do nothing.

last_record_lsn works, no? What's the difference?

Oh, now I think i get it. An open layer might contain more recent WAL that we are not going to flush to disk in this checkpoint. So iteratively, if we flush layer A that contained a record with LSN X, we must also flush any other layers that contain a record with LSN <= X, too. That's what the 'freeze_end_lsn' tracks.
If the workload consists of random updates, it's likely that we will end up flushing everything on every checkpoint, because every in-memory layer will contain records close the last_record_lsn.
Perhaps we should just always flush everything that's in memory. Would seem simpler. So the logic would be that if there are any in-memory layers with data older than checkpoint_distance, we flush everything to disk. Otherwise, we do nothing.

Yes, definitely in case of random updates all open layer should be flushed.
But I think that there are other popular workloads:

append-only tables
range partitioning
frequent creation of new tables and population them with data
...
when open layers have minimal intersection.
Such optimization costs almost nothing and doesn't significantly complicate code so  I think it should be done.
As far as I remember originally you have implemented some quite simple eviction algorithm and performance was very bad.
Then you have replace it with current selection of oldest layers and performance significantly improved.
I afraid that with naive eviction of all open layers we will get yet another noticeable performance degradation. Which is even now not so impressing.

Perhaps we should just always flush everything that's in memory. Would seem simpler. So the logic would be that if there are any in-memory layers with data older than checkpoint_distance, we flush everything to disk. Otherwise, we do nothing.

Yes, definitely in case of random updates all open layer should be flushed. But I think that there are other popular workloads:

append-only tables

range partitioning

frequent creation of new tables and population them with data
...
when open layers have minimal intersection.
Such optimization costs almost nothing and doesn't significantly complicate code so  I think it should be done.
As far as I remember originally you have implemented some quite simple eviction algorithm and performance was very bad.
Then you have replace it with current selection of oldest layers and performance significantly improved.
I afraid that with naive eviction of all open layers we will get yet another noticeable performance degradation. Which is even now not so impressing.

I think it would be ok. It's what happens with mostly random writes here anyway. But I'm OK with this PR too, with the last_lsn, if you prefer. Please add comments, though.

I am currently changing layers boundaries from [first_record.end_lsn, last_record.end_lsn+1)
to [first_record.start_lsn, last_record.end_lsn)
It seems to be much more natural, consistent with layer definition and correct (please notice that originally problem was caused by incorrect increment of disk_consistent_lsn using oldest_layer_lsn). But... looks like I opened Pandora's box:(
And yet another detected problem: we may now WAL insert record several times with the same LSN (because it affect multiple pages). This pages may belong to the same relation and same segment. As a result with can have in the layer several records  with the same LSN. What is worser, it can happen at layer boundary. In this case two segments will overlap:
prev_segment.end_lsn > next_segment.first_lsn. I am not sure that interval tree can correctly handle this case.
One of the solution is to assigned to each decoded block it own LSN (which will be somewhere inside WAL record).
Any better proposal?

On 26/01/2022 09:50, Konstantin Knizhnik wrote:

And yet another detected problem: we may now WAL insert record several
times with the same LSN (because it affect multiple pages). This pages
may belong to the same relation and same segment. As a result with can
have in the layer several records with the same LSN.

Sure.

What is worser, it can happen at layer boundary. In this case two
segments will overlap: prev_segment.end_lsn > next_segment.first_lsn.
I am not sure that interval tree can correctly handle this case.
Sorry, I didn't understand that. What's the problem?

Heikki

On 26/01/2022 09:50, Konstantin Knizhnik wrote: And yet another detected problem: we may now WAL insert record several times with the same LSN (because it affect multiple pages). This pages may belong to the same relation and same segment. As a result with can have in the layer several records with the same LSN.
Sure.
What is worser, it can happen at layer boundary. In this case two segments will overlap: prev_segment.end_lsn > next_segment.first_lsn. I am not sure that interval tree can correctly handle this case.
Sorry, I didn't understand that. What's the problem? - Heikki

We have segment with LSN range [100,200)
Next WAL record with LSN 210 touches several pages (contains N blocks), so we have to insert this record N times.
It can happen that touched pages belongs to this segment.
When we insert WAL record for first block, segment is changed to [100,210)
Then checkpoints happens and layer is frozen and evicted. As a result we have disk layer rel_*_100_210
Now we insert record for next block and have to create new in-memory layer: [200,inf)
Once it is evicted we will have to intersected delta layers, i.e. [100,210) and [200,300)

On 26/01/2022 11:36, Konstantin Knizhnik wrote:

We have segment with LSN range [100,200)
Next WAL record with LSN 210 touches several pages (contains N blocks),
so we have to insert this record N times.
It can happen that touched pages belongs to this segment.
When we insert WAL record for first block, segment is changed to [100,210)
Then checkpoints happens and layer is frozen and evicted. As a result we
have disk layer rel_*_100_210
Now we insert record for next block and have to create new in-memory
layer: [200,inf)
Once it is evicted we will have to intersected delta layers, i.e.
[100,210) and [200,300)

The writer that inserts the records holds the write lock on the
timeline, and it only advances last_record_lsn after it has inserted all
copies of the record. That should prevent the layer freezing from seeing
half-finished operations like that.

Heikki

I decided not to open Pandora's box now and leave existed layers boundaries.
I have to remove assert that oldest_pending  >= start_lsn because it is not true any more.
Bit all other layers handling logic is preserved.
And my restart test is passed.
I have also added comments.
Do you think that something else should be done in this PR.
I want to commit #1166 but its test sometimes not passed because of the problem fixed in this PR.
So it is better to commit this two PRs together.

I added a commit here that adds comments explaining the problem with the logic we had, and how 'last_lsn' solves it. Also, I renamed 'last_lsn' to 'tatest_lsn', and 'oldest_pending_lsn' to just 'oldest_lsn'. The 'oldest_lsn' and 'latest_lsn' are now symmetric.

I decided not to open Pandora's box now and leave existed layers boundaries.

Ok.

I have to remove assert that oldest_pending >= start_lsn because it is not true any more.

Wait, huh? How come?

I want to commit #1166 but its test sometimes not passed because of the problem fixed in this PR. So it is better to commit this two PRs together.

Makes sense.

Wait, huh? How come?

Because oldest_pending_lsn is start LSN of first record in layer and start_lsn is end LSN of first record in the layer.
oldest_pending_lsn is needed only for advancing disk_consistent_lsn. And we really need start LSN of record, otherwise will loose first record in non-evicted layer after restart.

Because oldest_pending_lsn is start LSN of first record in layer and start_lsn is end LSN of first record in the layer.

No, we don't use a record's start LSN anywhere. 'oldest_pending_lsn' is the end LSN of the first record in the layer.

oldest_pending_lsn is needed only for advancing disk_consistent_lsn. And we really need start LSN of record, otherwise will loose first record in non-evicted layer after restart.

That doesn't sound right. I believe that there is an off-by-one bug somewhere, and I can see that the tests fail if you revert those changes to how we set oldest_pending_lsn. But where exactly, I'm not sure.

That doesn't sound right. I believe that there is an off-by-one bug somewhere, and I can see that the tests fail if you revert those changes to how we set oldest_pending_lsn. But where exactly, I'm not sure.

The meaning of disk consistent lsn is that everything below it is already persistent in pageserver storage and so can be deleted from safekeepers. Disk consistent lsn is used as start replication point. Do sfekeeper starts sending WAL from this point. oldest_layer_lsn is used to advance disk_consistent_lsn. No objection till this point?
There was a bug that oldest_layer_lsn points to the end of record. So if we set disk_consistent_lsn to oldest_layer_lsn, then it means that this record is persisted. But actually it is not: it is still present only in opened in-memory layer and will be missed in case of restart.
So there are two solutions:

Do not use oldest_layer_lsn for advancing disk_consistent_lsn. This is what I tried to implement in previous PR, using end_lst of last evicted layer. But you explained me why this approach cause starvation in case of not synchronized layers freeze LSNs.
Make oldest_layer_lsn  point to the beginning of record rather then to the end of record. As far as oldest_layer_lsn is used only for adjusting disk_consistent_lsn, it will not cause problems with any other parts of code.

So I find your original implementation of calculating next disk_consistent_lsn simpler and more straightforward than  1). We in any case need to synchronize freeze LSNs. And this is what is done in this PR.
I do not understand why it sounds wrong. What is wrong from my point of view is current definition of layer boundaries: [first_record.end_lsn...last_record_end_lsn+1)
It contradicts to what is written in comments:
    ///
    /// This layer contains all the changes from 'start_lsn'. The
    /// start is inclusive.
    ///

IMHO changing layer boundaries to [start_record.first_lsn, last_record.last_lsn) is my natural and clear.
It mostly eliminate need in +-1 manipulations with end_lsn and confusions whether it is inclusive or exclusive, i.e. code like this:
       let end_lsn_exclusive = inner.end_lsn.unwrap();
        let end_lsn_inclusive = Lsn(end_lsn_exclusive.0 - 1);

I tried to make this change and spent half of the day trying to make it work.
But then I decided that making such dangerous change just before launch is not good idea
and decided to make minimal possible patch. Changing oldest_layer_lsn definition and removing one assert seems to be simplest solution of the problem.

On 26/01/2022 23:25, Konstantin Knizhnik wrote:

That doesn't sound right. I believe that there is an off-by-one bug
somewhere, and I can see that the tests fail if you revert those
changes to how we set oldest_pending_lsn. But where exactly, I'm not
sure.

The meaning of disk consistent lsn is that everything below it is
already persistent in pageserver storage and so can be deleted from
safekeepers. Disk consistent lsn is used as start replication point. Do
sfekeeper starts sending WAL from this point. oldest_layer_lsn is used
to advance disk_consistent_lsn. No objection till this point?

Ok.

There was a bug that oldest_layer_lsn points to the end of record. So if
we set disk_consistent_lsn to oldest_layer_lsn, then it means that this
record is persisted. But actually it is not: it is still present only in
opened in-memory layer and will be missed in case of restart.

Hmm, ok.

So there are two solutions:

Do not use oldest_layer_lsn for advancing disk_consistent_lsn. This
is what I tried to implement in previous PR, using end_lst of last
evicted layer. But you explained me why this approach cause
starvation in case of not synchronized layers freeze LSNs.
Make oldest_layer_lsn point to the beginning of record rather then
to the end of record. As far as oldest_layer_lsn is used only for
adjusting disk_consistent_lsn, it will not cause problems with any
other parts of code.

So I find your original implementation of calculating next
disk_consistent_lsn simpler and more straightforward than 1). We in any
case need to synchronize freeze LSNs. And this is what is done in this PR.
I do not understand why it sounds wrong. What is wrong from my point of
view is current definition of layer boundaries:
[first_record.end_lsn...last_record_end_lsn+1)
It contradicts to what is written in comments:
|/// /// This layer contains all the changes from 'start_lsn'. The ///
start is inclusive. /// |

It's not wrong as such. All the LSNs refer to a record's end LSN. If you
read it like that, it's consistent.
I understand now that it's causing trouble with setting
disk_consistent_lsn. If you take the oldest LSN of the first layer
that's not stored durably to disk, and try to start streaming
replication from that LSN, you will skip a record. Because the oldest
LSN is an end LSN, and for replication, you need the start LSN of the
record. We just don't currently store that information anywhere.

IMHO changing layer boundaries to [start_record.first_lsn,
last_record.last_lsn) is my natural and clear.
It mostly eliminate need in +-1 manipulations with end_lsn and
confusions whether it is inclusive or exclusive, i.e. code like this:
|let end_lsn_exclusive = inner.end_lsn.unwrap(); let end_lsn_inclusive =
Lsn(end_lsn_exclusive.0 - 1); |

Yeah, I see that now. Makes sense.

I tried to make this change and spent half of the day trying to make it
work.
But then I decided that making such dangerous change just before launch
is not good idea
and decided to make minimal possible patch. Changing oldest_layer_lsn
definition and removing one assert seems to be simplest solution of the
problem.

It just feels very strange to me that the oldest_lsn can be smaller than
the start_lsn. Agreed that that's probably the smallest fix right now.
Another straightforward solution is to flush all in-memory layers, if
you're going to flush any of them. That way, disk_consistent_lsn is
always set to last_record_lsn, not to the oldest LSN of any layer.)

Heikki

It's not wrong as such. All the LSNs refer to a record's end LSN. If you
read it like that, it's consistent.

Well, this is an issue of agreements.
But LSN is a point, while WAL record is  a range of LSNs. And it is the point of confusion.

It just feels very strange to me that the oldest_lsn can be smaller than
the start_lsn.

+1
This is why I first of all decided to change start_lsn as well. But unfortunately it cause cascade of changes and I failed to make it work in some reasonable amount of time.

Another straightforward solution is to flush all in-memory layers, if
you're going to flush any of them.

Yes, it is true.
As I already wrote, I afraid that eviction of all layers at each checkpoint will cause significant degrade of performance (especially with 1 second checkpoint interval). At lest we should handle case when no layer is evicted at all because none of them cross checkpoint distance.
So, right now this PR is passing tests including my shell script which performs restarts after killing all postgres/pageserver/safekeeper processes. Should we commit it as it is or you prefer more straightforward solution (evict all layers)? In this case we can  throw away oldest_layer_lsn at all because it is not needed for any other purpose... Please also notice that this "straightforward solution" is not simpler and also requires two loops: first one to check if any layer cross checkpoint distance and if so, then second loop should evict all open layers.

Yes, it is true. As I already wrote, I afraid that eviction of all layers at each checkpoint will cause significant degrade of performance (especially with 1 second checkpoint interval). At lest we should handle case when no layer is evicted at all because none of them cross checkpoint distance.

Yeah, checking for checkpoint distance is important.

So, right now this PR is passing tests including my shell script which performs restarts after killing all postgres/pageserver/safekeeper processes. Should we commit it as it is or you prefer more straightforward solution (evict all layers)?

Let's commit this. Please check and fix the comments, I think some of the comments I added are not accurate, now that oldest_lsn is the record's start LSN.

In this case we can throw away oldest_layer_lsn at all because it is not needed for any other purpose... Please also notice that this "straightforward solution" is not simpler and also requires two loops: first one to check if any layer cross checkpoint distance and if so, then second loop should evict all open layers.

Well, the layers are in oldest-first order, so you can peek at the first one, and check if it crosses checkpoint distance. And if it does, evict all of them. You could also get rid of the "generation" stuff there.
But since PR works, let's just get it in.

@hlinnaka  are there some other comments I need to fix?


Parent's Contact number to be taken

Kindly add parents' contact number also.

corrected in #002545 check it out
npm install and check should work now

Thanks . !


Check if switching to query_interface instead of our own minimal version of it makes sense

https://docs.rs/query_interface/0.3.5/query_interface/
Could simply quite some code and potentially makes the implementation of interfaces simpler (no need for get_impl).

It doesn't, we don't make use of it anymore at all


Issue 1131 | Drop keycloak

Summary
Drop keycloak
Details and comments
Closes #1131

Before merging, I'd like to test this against local Kubernetes too... (I'll run on kind once the PR is out of draft)

I had almost the same so I'm fine with this PR, @IceKhan13 . What I would do is simplify it a bit, let keycloak stuff and remove everything in another PR, what do you think?

We have tons of configurations related with keycloak that it will be better review in apart from this I think.

We have tons of configurations related with keycloak that it will be better review in apart from this I think.

Do you mean configurations in helm charts?

What I would do is simplify it a bit

most of the changes in just notebooks diffs :)
if you look at code lines it will be like 50 lines of change, which is simple, I would say 😄
I do not mind if you close this PR and push your changes, I was just sitting in a meeting and decided to take this one.

Do you mean configurations in helm charts?

Mostly yeah, but around applications too. Something that we can clean in the next PR.

most of the changes in just notebooks diffs :)
if you look at code lines it will be like 50 lines of change, which is simple, I would say 😄
I do not mind if you close this PR and push your changes, I was just sitting in a meeting and decided to take this one.

Yeah, no it's fine. Just to mention in case you were thinking to include to remove all the configuration stuff. That maybe we can let it for another PR.

Should we remove this from client too, @IceKhan13 ?:
https://github.com/Qiskit-Extensions/quantum-serverless/blob/main/client/quantum_serverless/core/provider.py#L455

Oh, let me remove from client to. YOu are right.

Got an error when running on Kind:
QuantumServerlessException: Cannot verify token.
(did a pip install . on client, so should be using the most recent updates...)

Did you change the authentication mechanism

No, wasn't expecting that I'd need to (I assumed it would work out of the box)... let me try again

hmmm... still fails with
 {
    "name": "SETTINGS_AUTH_MECHANISM",
    "value": "mock_token"
  },

@psschwei did you rebuild gateway image?

@psschwei did you rebuild gateway image?

yes, built from source

envvar is being picked up inside the gateway pod...
I have no name!@gateway-7b444889b6-dkjh8:/usr/src/app$ python
Python 3.9.16 (main, May 23 2023, 14:24:31)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ.get("SETTINGS_AUTH_MECHANISM")
'mock_token'
>>>

🤔

looks like the keycloak pod is still there... let me try deleting that (or maybe I should wait for the helm config update in the next PR...)

keycloak chart should not affect it by any means

does the postgres username matter? looks like I was using bn_keycloak, but not sure if that user is gone now...

nope, it does not matter

Helm should continue being the same with the difference in the auth.mechanism value I think 🤔

Ok, I figured out the error...  for reference, this was the code block:
serverless = ServerlessProvider(
    token=os.environ.get("GATEWAY_TOKEN", "awesome_token"),
    host=os.environ.get("GATEWAY_HOST", "http://localhost:8010"),
)
serverless

Can anyone spot the problem? :smile:

Damn port?

Damn port?

You win :laughing:

ref: https://github.com/Qiskit-Extensions/quantum-serverless/issues/1133

Tested it with rancher locally too. Got the same error like Paul :)

Tomorrow I can upload helm configuration.

Perfect!


feat: create an option to sort comments according to likes/date-added

Is your feature request related to a problem? Please describe
The default sorting for comments should be based on the number of likes, with an additional option available to sort them by the date they were added if desired.

Describe the solution you'd like
Similar to the image shown above, let's incorporate a sorting button on the top right.
Describe alternatives you've considered
No response
Additional context
No response
Are you working on this?
Yes

Hi @NayakPenguin , you can work on the issue.
You may keep recent comments as default sorting mechanism like now.

Hey @NayakPenguin , are you working on this issue.

I wasn't able to resolve it, I would greatly appreciate you to pick it up @Mayank77maruti

Sure @NayakPenguin , I would love to work on it. Thank you for letting me know.

Sure @NayakPenguin , I would love to work on it. Thank you for letting me know.

Go ahead !

Hi @tanmoysrt  , I am facing issue in running the repo locally hereby i am attaching some screenshots please help me.

and when i do bundle install

please do help me how do i resolve this issue and run the repo locally.

Hi @tanmoysrt  , I am facing issue in running the repo locally hereby i am attaching some screenshots please help me.

and when i do bundle install

please do help me how do i resolve this issue and run the repo locally.

Try to restart the docker containers.

hey @Mayank77maruti any update about the devolpment for this issue??


Allow optional parameters for read_xlsx

Allows user to pass optional parameters to read_xlsx for greater control over annotation file read in.
Resolves #111

Reviewed and approved added test cases.


Use ION-separated AMP Manager and Agent

An APL IRAD project separated the AMP Agent and Manager from the ION source tree, with shim behavior to retain existing ION application behavior.
The project is hosted on Github at DTNMA-Tools and could be added to or forked into the ANMS to enable tracking AMP/ADM document updates. Those document updates haven't fully stabilized, so this is a future enhancement ticket.

@d-linko Should this be closed by the merging of #79?


instance.reschedule_workflow fails on "destroy never"

Run following sequence of commands:
instance.reschedule_workflow(workflow_name="destroy", timestamp=3600)
instance.reschedule_workflow(workflow_name="destroy", timestamp=-1)
instance.reschedule_workflow(workflow_name="destroy", timestamp=3600)

Last call will fail with:
  File "qubell/api/private/instance.py", line 392, in reschedule_workflow
    workflow_id = [x['id'] for x in self.scheduledWorkflows if x['name'] == workflow_name][0]
IndexError: list index out of range

Expected behaviour - workflow is rescheduled from never to given time. This is expected because on UI this is the same operation.

Looks like you should use schedule_workflow. I'm not sure it's client's lib responsibility, while different routes with different parameters used. So, best solution here is throw error with explaining text.
Or add hidden logic - schedule workflow if it does not exists.
I'd prefer first one


Use Uint8Array instead of Buffer in src/protobuf

This should allow the src/protobuf to be lifted out of masterchat and put into its own library for easy consumption elsewhere (including in browsers)

All tests passed! Thank you for the awesome job 🎉


Features/feature name validation

Thanks for submitting a PR! Please check the boxes below:

[x] I have run pre-commit to check linting
[x] I have filled in the "Changes" section below?
[x] I have filled in the "How did you test this code" section below?

Adding RegEx Validation
https://user-images.githubusercontent.com/8608314/208103125-81afd51a-388f-4738-8cbf-19dfb4076195.mov
Only allowing valid RegEx's

Using this PR as a place to put this comment but I feel like this is an API issue more than a FE issue.
I tested this using the regex string [a-z]+ and then was able to create a feature called test_feature, whereas I was expecting it to fail. The issue being that I hadn't explicitly stated that I wanted the regex to match the entire string (i.e. ^[a-z]+$).
The question, however, is whether we want to automatically wrap the regex string in ^ and $ to ensure people don't get unusual behaviour like this? Or are we happy giving them full control, but perhaps adding wording that they should wrap their regex in these characters?
Also, I noticed that there is an extra $ being prepended to the regex string in the UI warning on the feature create page which seems a bit misleading.

I think it would be nice, as an improvement to add a 'tester' input next to the input on the project settings page. Not essential but it would be nice.

Added


Update books to current state

Deploy preview for openbuildservice ready!

Could we automate this by running the script update_documentation.sh every X day/week/whenever we need?
How often do we need to update the books?

Could we automate this by running...
How often do we need to update...

This should happen as soon as there is a new commit in openSUSE/obs-docu#master. The "problem" is that this is using openSUSE/daps which is not a very popular build tool so it's not easily injected into tools like netlify.
I just added a docker dev-env and travis to obs-docu. Maybe we can get a deploy task going that pushed to master in this repo...

dmarcoux commented 3 hours ago
A bunch of questions from my naive newcomer point of view.Could we automate this by running the script update_documentation.sh every X day/week/whenever we need?

We could and we should!
We could have another bot in our tools repo. Then whenever the master branch of obs-docu changes, our bot would receive a webhook, spins up the docker image, updates the docs and creates a PR.
We just need someone to do it:smile:

How often do we need to update the books?

Not that often. You could check the PRs to get an idea. AFAIK it's every few month.

@openSUSE/obs-frontend Please review


Client application startup is very slow

On the SAGE cluster with 23 nodes (7 server and 16 client nodes with 16 m0_client processes configured each) the startup of any client application (c0cp or mcp) takes about 20-30 seconds. And the more m0_client processes are configured - the more time it takes to start up. (That's why the number of configured m0_client processes is limited to 16 atm.)
See attached m0trace.19259.txt.zip from the client run.

@mssawant, @knekrasov any idea why Preparing the reply for HaNvecGetEvent takes so much time?

@andriytk

any idea why Preparing the reply for HaNvecGetEvent takes so much time?

I think the reason is that this preparation caused more than 3000 separate read requests to Consul KV.
Here are some details:
This is the handler method:
   @log_exception
    def ha_nvec_get_reply(self, event: HaNvecGetEvent) -> None:
        LOG.debug('Preparing the reply for HaNvecGetEvent (nvec size = %s)',
                  len(event.nvec))
        notes: List[HaNoteStruct] = []
        for n in event.nvec:
            n.note.no_state = self.consul_util.get_conf_obj_status(
                ObjT[n.obj_t], n.note.no_id.f_key)
            notes.append(n.note)

        LOG.debug('Replying ha nvec of length ' + str(len(event.nvec)))
        self._ffi.ha_nvec_reply(event.hax_msg, make_array(HaNoteStruct, notes),
                                len(notes))

According to the log you provided, the nvec length is 1024. So the loop above had 1024 iterations; self.consul_util.get_conf_obj_status() was invoked that many times.
If we go down to get_conf_obj_status(), we'll see 2 explicit KV reads there:
@repeat_if_fails()
    def get_conf_obj_status(self, obj_t: ObjT, fidk: int) -> int:
        obj_state: int = HaNoteStruct.M0_NC_ONLINE
        if obj_t.name in (ObjT.PROCESS.name, ObjT.SERVICE.name):
            # 'node/<node_name>/process/<process_fidk>/service/type'
            node_items = self.kv.kv_get('m0conf/nodes', recurse=True)
            # TODO [KN] This code is too cryptic. To be refactored.
            keys = getattr(self,
                           'get_{}_keys'.format(
                                obj_t.name.lower()))(node_items, fidk)
            assert len(keys) == 1
            key = keys[0].split('/')
            node_key = ('/'.join(key[:3]))
            node_val = self.kv.kv_get(node_key)
            data = node_val['Value']
            node_name: str = json.loads(data)['name']
            if (self.get_node_health(node_name) != 'passing'):
                obj_state = HaNoteStruct.M0_NC_FAILED

        if obj_t.name in (ObjT.PROCESS.name, ObjT.SERVICE.name):
            obj_state = self.get_proc_svc_conf_obj_status(obj_t, fidk)
        elif obj_t.name in (ObjT.SDEV.name, ObjT.DRIVE.name):
            obj_state = self.get_sdev_state(obj_t, fidk)

        return obj_state

Plus - indirectly - one more KV call when ObjT is PROCESS:

    def get_service_process_fid(self, svc_fid: Fid) -> Fid:
        assert ObjT.SERVICE.value == svc_fid.container
        node_items = self.kv.kv_get('m0conf/nodes', recurse=True)
        keys = self.get_service_keys(node_items, svc_fid.key)
        assert len(keys) == 1
        process_fid: str = keys[0].split('/')[4]
        pfid = Fid.parse(process_fid)
        return pfid

Please also note that as of #1612 Motr-related activity is hax is parallelized. It means that although it will not speed this particular HaNvecGet processing up, other clients will not be blocked by this processing. So I think that if you take fresher Hare from main branch, the overall performance will be better.

@knekrasov thanks for this analysis! Yes, I noticed that consul and hax would consume a lot of CPU during this time.
Am I correctly understand that the single process performance is not improved in the new Hare code?

I'm not sure I understand what "single process" means.
The optimization that #1612 is as follows: If we have 2 HaNvecGet, the second HaNvecGet may be processed earlier, than the first one is fully processed. In general case, Hax will parallelize all Motr-related tasks unless they don't "conflict" (sometimes they do and then ordering takes place).
That means that if you have a heavy HaNvecGet, that request will still be heavy in a newer Hare version. But there is a good chance that overall (effective) performance will still be better (other clients will not be blocked until a heavy request is being processed).

Sorry for confusion, I meant single client. Got it.
I think we need to optimise the single HaNvecGet processing as well somehow. This is what this issue is about.

Is it possible to make one -recurse request to Consul from hax and parse the result?

I think it is feasible but this requires a good design. I would suggest to start thinking about a query DSL (similar to DOM manipulation API that jQuery has). The DSL can describe (in an FP manner) what we want to do with KV and how do we want to traverse the KV tree. The executor of such a DSL can optimize the amount of KV calls - just like RDBMS do - because the whole query is visible in a single place and the executor fully understands the semantics.
Without that DSL we have multiple functions, each of them may read from KV (perhaps with recurse) and delegate part of the logic to other functions - that also may read from KV. This is totally fine in terms of functions reusability but blocks us from the query optimization.

@knekrasov I see that get_conf_obj_status() and get_service_process_fid() already do the recursive queries on m0conf/nodes. So calling them in the loop for each cluster entity (note.no_id) is very expensive.
Can we instead make the recursive query one time outside these functions and pass node_items as a parameter to them?

@andriytk, @knekrasov, yes, presently more number of Consul requests are generated because we have individual keys per object instead of a list as the values for a given parent. So we fetch all the keys related to a given object and then parse the result to filter out the ones not needed. Have created a ticket sometime back to optimize the KV structure a bit by having list of object as values for a given key that would reduce the number of Consul KV requests. I think we need to schedule that task.

presently more number of Consul requests are generated because we have individual keys per object instead of a list as the values for a given parent. So we fetch all the keys related to a given object and then parse the result to filter out the ones not needed.

@mssawant Please be careful with such decisions 😄 What you described is a de-normalization. Which is a risk for rolling upgrade. As you remember, for seamless rolling upgrades it is safer to prefer individual keys with atomic values to JSON-encoded values with internal structure.

@knekrasov, Yeah, updating schema is one part but as you and @andriytk pointed out, we have too many recursive consul calls to m0conf/nodes. I did try to capture the paths and functions, I temporarily added a wrapper call to fetch m0conf/nodes keys to help find the callers and there seem to be a lot but the paths mostly are through broadcast_ha_states() (and few others).
_process_event_cb() -> fid_to_endpoint() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

ProcessEvent -> _do_work() -> _update_process_status() -> broadcast_ha_states() -> _generate_sub_services() -> get_services_by_parent_process() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
ProcessEvent -> _do_work() -> _update_process_status() -> broadcast_ha_states() -> update_drive_state() -> set_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> get_services_by_parent_process() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> update_drive_state() -> set_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> get_services_by_parent_process() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> update_drive_state() -> set_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

EntrypointRequest -> _entrypoint_request_cb() -> is_proc_client() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
HaNvecGetEvent -> ha_nvec_get_reply() -> get_conf_obj_status() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

ProcessEvent -> _do_work() -> _update_process_status() -> broadcast_ha_states() -> notify_node_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
ProcessEvent -> _do_work() -> _update_process_status() -> broadcast_ha_states() -> notify_node_status() -> get_node_ctrl_fid() -> get_node_encl_fid() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
ProcessEvent -> _do_work() -> _update_process_status() -> broadcast_ha_states() -> notify_node_status() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
ProcessEvent -> _do_work() -> _update_process_status() -> broadcast_ha_states() -> notify_node_status() -> get_node_encl_fid() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_node_ctrl_fid() -> get_node_encl_fid() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_node_encl_fid() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_node_ctrl_fid() -> get_node_encl_fid() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> notify_node_status() -> get_node_encl_fid() -> get_node_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

ProcessEvent -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> _generate_sub_disks() -> update_drive_state() -> set_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> _generate_sub_disks() -> update_drive_state() -> set_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> _generate_sub_disks() -> update_drive_state() -> set_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

ProcessEvent -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> _generate_sub_disks() -> get_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
FirstEntrypointRequest -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> _generate_sub_disks() -> get_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
BroadcastHAStates -> _do_work() -> broadcast_ha_states() -> _generate_sub_services() -> _generate_sub_disks() -> get_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

HaNvecGetEvent -> ha_nvec_get_reply() -> get_conf_obj_status() -> get_sdev_state() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

handle_device_state_set() -> to_ha_state() -> drive_to_sdev_fid() -> node_to_drive_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

broadcast_ha_states() -> update_drive_state() -> drive_to_sdev_fid() -> node_to_drive_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
_generate_sub_services() -> _generate_sub_disks() -> update_drive_state() -> drive_to_sdev_fid() -> node_to_drive_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

get_sdev_state() -> drive_to_sdev_fid() -> node_to_drive_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
drive_name_to_id() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

ensure_ioservices_running() -> get_m0d_statuses() -> get_process_node -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
m0ds_stopping() -> get_m0d_statuses() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
get_svc_status() -> get_process_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
get_proc_svc_conf_obj_status() -> get_process_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
get_service_health() -> get_process_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
is_confd_failed() -> get_process_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
get_m0d_statuses() -> get_service_health() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
get_process_current_status() -> get_service_health() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
update_process_failure() -> get_process_current_status() -> get_process_node() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)
get_conf_obj_status() -> get_proc_svc_conf_obj_status() -> get_service_process_fid() -> get_all_nodes_keys() -> kv_get('m0conf/nodes', recurse=True)

We need to surely reduce these but I think this list will help us identify a one or more points at which we need to fetch all the m0conf/nodes keys and reduce the number of Consul calls.

@andriytk, @knekrasov kindly review  https://github.com/Seagate/cortx-hare/pull/1737 as time permits.

I suggest doing the following: on "Preparing the reply for HaNvecGetEvent" 1) call self.kv.kv_get('m0conf/nodes', recurse=True) only once, and 2) parse the result into some hash table (dictionary in Python language). 3) Use the resulting dictionary to construct the reply.

@andriytk Thanks for the comment. The thing is that is exactly how this caching mechanism works already.
Heavyweight self.kv.kv_get('m0conf/nodes', recurse=True) are executed only ONCE during the whole nvec_get reply procedure, the result is memoized in the cache instance and afterwards they are taken from the cache. The decision whether a heavyweight call needs to be executed or the pre-calculated result can be taken directly from cache is made at the level of @uses_consul_cache decorator.

The numbers are better, but still - 10 seconds is very far from adequate time. It should be less than 1 sec.

Vaibhav Prakash Paratwar commented in Jira Server:

Closing as EOS-22148 resolved.

Vaibhav Prakash Paratwar commented in Jira Server:

.

Vaibhav Prakash Paratwar commented in Jira Server:

.


After optimization the width or height value changes

Describe the bug
With some SVG files, after the optimization with SVGO, the SVG width or height attributes change their value, resulting in a different rendering.
To Reproduce
Steps to reproduce the behavior:
I try to optimize the following file with the command yarn run svgo --pretty test.svg
<svg xmlns="http://www.w3.org/2000/svg" width="127mm" height="61mm" viewBox="0 0<PHONE_NUMBER>"><line x1="9.9213" y1="170.0787" x2="350.0787" y2="170.0787" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="2.8346" y1="162.9921" x2="2.8346" y2="9.9213" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="9.9213" y1="2.8346" x2="350.0787" y2="2.8346" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="357.1654" y1="162.9921" x2="357.1654" y2="9.9213" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><path d="M357.1654,9.9213a7.0867,7.0867,0,0,0-7.0867-7.0867" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><path d="M350.0787,170.0787a7.0866,7.0866,0,0,0,7.0867-7.0866" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><path d="M2.8346,162.9921a7.0866,7.0866,0,0,0,7.0867,7.0866" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><path d="M9.9213,2.8346A7.0867,7.0867,0,0,0,2.8346,9.9213" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="323.1496" y1="65.1969" x2="323.1496" y2="107.7165" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="317.4803" y1="107.7165" x2="317.4803" y2="65.1969" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><path d="M15.5977,99.9626l2.1191.6992v.8994l-6.7363-2.2881v-1.05l6.7363-2.2988v.93l-2.1191.72Zm-.68-2.209-1.9385.66c-.44.1494-.84.249-1.2295.35v.02c.39<IP_ADDRESS>,1.22.34l1.9483.66Z"/><path d="M17.4971,90.3747a4.3094,4.3094,0,0,1,.3193,1.7783,3.1477,3.1477,0,0,1-3.4082,3.3281A3.3524,3.3524,0,0,1,10.87,91.9626a3.6221,3.6221,0,0,1,.3-1.5987l.71.21a3.1621,3.1621,0,0,0-.2793,1.3594,2.5046,2.5046,0,0,0,2.7783,2.6289,2.449,2.449,0,0,0,2.708-2.5889,3.4775,3.4775,0,0,0-.2793-1.419Z"/><path d="M10.9805,86.2926h6.7363v.8692H10.9805Z"/><path d="M17.7168,84.7716H10.9805v-.9492l3.4082-2.1587a18.9925,18.9925,0,0,1,2.1884-1.2094l-.01-.02c-.8994.08-1.7188.1-2.7686.1H10.9805v-.82h6.7363v.8794l-3.418,2.1392a21.4393,21.4393,0,0,1-2.249,1.2592l.01.03c.85-.05,1.6592-.07,2.7783-.07h2.8789Z"/><path d="M339.8887,105.776a12.43,12.43,0,0,1-.14-1.8486,3.8564,3.8564,0,0,1,.84-2.7285,3.1006,3.1006,0,0,1,2.4179-.9492,3.5673,3.5673,0,0,1,2.6192.9687,4.1548,4.1548,0,0,1,.9687,2.9785,14.0985,14.0985,0,0,1-.0791,1.5791Zm5.957-.8691a5.4946,5.4946,0,0,0,.05-.88,2.5889,2.5889,0,0,0-2.8584-2.8682,2.4027,2.4027,0,0,0-2.5986,2.7286,4.95,4.95,0,0,0,.09,1.0195Z"/><path d="M346.3154,94.41a4.3082,4.3082,0,0,1,.3194,1.7783,3.1478,3.1478,0,0,1-3.4082,3.3282,3.3524,3.3524,0,0,1-3.5381-3.5186,3.62,3.62,0,0,1,.3-1.5986l.71.21a3.1639,3.1639,0,0,0-.2793,1.3594,2.5047,2.5047,0,0,0,2.7784,2.6289,2.449,2.449,0,0,0,2.708-2.5888,3.4771,3.4771,0,0,0-.2793-1.419Z"/><path d="M343.0967,85.44c2.3193,0,3.5488,1.41,3.5488,3.1284,0,1.7793-1.38,3.0283-3.4189,3.0283-2.1387,0-3.5381-1.3291-3.5381-3.1279C339.6885,86.6286,341.0977,85.44,343.0967,85.44Zm.11,5.228c1.44,0,2.7285-.78,2.7285-2.1494,0-1.38-1.2695-2.1592-2.7988-2.1592-1.3388,0-2.7383.6993-2.7383,2.1485C340.3984,89.9469,341.7275,90.6676,343.207,90.6676Z"/><path d="M339.7988,83.4372h3.9883c1.5088,0,2.1484-.67,2.1484-1.5694,0-.9995-.66-1.6391-2.1484-1.6391h-3.9883v-.8794h3.9278c2.0693,0,2.9189,1.0893,2.9189,2.5483,0,1.3794-.79,2.419-2.8789,2.419h-3.9678Z"/><path d="M340.5381,76.5573v2.0488h-.7393V73.6193h.7393v2.0586h5.9971v.8794Z"/><line x1="314.6457" y1="52.4409" x2="317.4803" y2="65.1969" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="314.6457" y1="52.4409" x2="323.1496" y2="65.1969" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="323.1496" y1="107.7165" x2="325.9843" y2="120.4724" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><line x1="317.4803" y1="107.7165" x2="325.9843" y2="120.4724" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/></svg>

and I'm getting this result:
<svg xmlns="http://www.w3.org/2000/svg" width="480" height="61mm" viewBox="0 0 360 172.913">
  <path fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round" d="M9.921 170.079h340.158M2.835 162.992V9.921M9.921 2.835h340.158M357.165 162.992V9.921M357.165 9.921a7.087 7.087 0 0 0-7.086-7.086M350.079 170.079a7.087 7.087 0 0 0 7.086-7.087M2.835 162.992a7.087 7.087 0 0 0 7.086 7.087M9.921 2.835A7.087 7.087 0 0 0 2.835 9.92M323.15 65.197v42.519M317.48 107.716V65.197"/>
  <path d="m15.598 99.963 2.119.699v.9l-6.736-2.289v-1.05l6.736-2.299v.93l-2.12.72Zm-.68-2.21-1.939.66c-.44.15-.84.25-1.23.35v.02c.<IP_ADDRESS> 1.22.34l1.949.66ZM17.497 90.375a4.31 4.31 0 0 1 .32 1.778 3.148 3.148 0 0 1-3.409 3.328 3.352 3.352 0 0 1-3.538-3.518 3.622 3.622 0 0 1 .3-1.6l.71.21a3.162 3.162 0 0 0-.28 1.36 2.505 2.505 0 0 0 2.779 2.63 2.449 2.449 0 0 0 2.708-2.59 3.478 3.478 0 0 0-.28-1.419ZM10.98 86.293h6.737v.869H10.98ZM17.717 84.772H10.98v-.95l3.408-2.158a18.992 18.992 0 0 1 2.188-1.21l-.01-.02c-.9.08-1.719.1-2.769.1H10.98v-.82h6.737v.88l-3.418 2.139a21.44 21.44 0 0 1-2.25 1.26l.01.03c.85-.05 1.66-.07 2.78-.07h2.878ZM339.889 105.776a12.43 12.43 0 0 1-.14-1.849 3.856 3.856 0 0 1 .84-2.728 3.1 3.1 0 0 1 2.418-.95 3.567 3.567 0 0 1 2.619.97 4.155 4.155 0 0 1 .968 2.978 14.098 14.098 0 0 1-.079 1.579Zm5.957-.87a5.495 5.495 0 0 0 .05-.88 2.589 2.589 0 0 0-2.859-2.867 2.403 2.403 0 0 0-2.598 2.728 4.95 4.95 0 0 0 .09 1.02ZM346.315 94.41a4.308 4.308 0 0 1 .32 1.778 3.148 3.148 0 0 1-3.408 3.328 3.352 3.352 0 0 1-3.539-3.518 3.62 3.62 0 0 1 .3-1.599l.71.21a3.164 3.164 0 0 0-.279 1.36 2.505 2.505 0 0 0 2.779 2.629 2.449 2.449 0 0 0 2.708-2.59 3.477 3.477 0 0 0-.28-1.418ZM343.097 85.44c2.319 0 3.549 1.41 3.549 3.128 0 1.78-1.38 3.029-3.42 3.029-2.138 0-3.538-1.33-3.538-3.128 0-1.84 1.41-3.029 3.409-3.029Zm.11 5.228c1.44 0 2.728-.78 2.728-2.15 0-1.38-1.27-2.159-2.799-2.159-1.338 0-2.738.7-2.738 2.149 0 1.439 1.33 2.16 2.809 2.16ZM339.799 83.437h3.988c1.509 0 2.149-.67 2.149-1.57 0-.999-.66-1.638-2.149-1.638H339.8v-.88h3.928c2.069 0 2.919 1.09 2.919 2.549 0 1.379-.79 2.419-2.88 2.419H339.8ZM340.538 76.557v2.05h-.74v-4.988h.74v2.059h5.997v.88Z"/>
  <path fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round" d="m314.646 52.441 2.834 12.756M314.646 52.441l8.504 12.756M323.15 107.716l2.834 12.756M317.48 107.716l8.504 12.756"/>
</svg>

The width value is now 480, but it was 127mm.
The same happens for few other files, but hundreds of other files are fine.
Expected behavior
The width and height values should not change.
Desktop (please complete the following information):

SVGO Version 2.8.0
NodeJs Version v16.14.0
OS: MacOS 12.2.1

Additional context
I'm using svgo with yarn inside a Ruby on Rails application in development to optimize my assets.
My SVGO configuration (svgo.config.js):
module.exports = {
  plugins: [
    {
      name: 'preset-default',
      params: {
        overrides: {
          removeViewBox: false,
        }
      }
    },
    'prefixIds'
  ],
  js2svg: {
    indent: 2, // string with spaces or number of spaces. 4 by default
    pretty: true, // boolean, false by default
  }
}

I found something that the problematic files have in common is that in the viewbox attribute the corresponding value for the width or height (in this case 360) has no decimal point and the other value is decimal (in this case 172.9134).
I don't know if it is by chance or it could be help to find the problem.
Thank you very much for your support.

I just found a very similar issue, I don't know if it could be related: https://github.com/svg/svgo/issues/1080

Yeah, cleanupNumericValues is the culprit here too, so it seems your issue is a duplicate of #1080.

Thank you for the reply.
I close the ticket as duplicate.


Fail to run test with node 16.1.0

Describe the bug
It throws unhandled error.
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: Initiated Worker with invalid execArgv flags: -C
 ❯ new NodeError node:internal/errors:363:5
 ❯ new Worker node:internal/worker:200:13
 ❯ ThreadPool._addNewWorker node_modules/tinypool/dist/esm/index.js:500:20
 ❯ ThreadPool._ensureMinimumWorkers node_modules/tinypool/dist/esm/index.js:485:12
 ❯ new ThreadPool node_modules/tinypool/dist/esm/index.js:475:10
 ❯ new Tinypool node_modules/tinypool/dist/esm/index.js:739:31
 ❯ createPool node_modules/vitest/dist/chunk-vite-node-externalize.e66d46f6.js:7167:16
 ❯ node_modules/vitest/dist/chunk-vite-node-externalize.e66d46f6.js:9998:21
 ❯ Vitest.runFiles node_modules/vitest/dist/chunk-vite-node-externalize.e66d46f6.js:10015:7
 ❯ Vitest.start node_modules/vitest/dist/chunk-vite-node-externalize.e66d46f6.js:9932:5

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: {
  "code": "ERR_WORKER_INVALID_EXEC_ARGV",
}

Reproduction
I downloaded stackblitz sample vitest project and run it in node 16.1.0 without any change.
https://stackblitz.com/fork/github/vitest-dev/vitest/tree/main/examples/basic?initialPath=__vitest__
System Info
System:
  OS: macOS 12.4
  CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
  Memory: 4.38 GB / 32.00 GB
  Shell: 5.8.1 - /bin/zsh
Binaries:
  Node: 16.1.0 - ~/.asdf/installs/nodejs/16.1.0/bin/node
  Yarn: 1.22.17 - ~/.asdf/installs/nodejs/16.1.0/.npm/bin/yarn
  npm: 7.11.2 - ~/.asdf/plugins/nodejs/shims/npm
Browsers:
  Chrome: 107.0.5304.87
  Safari: 15.5
npmPackages:
  vite: latest => 3.2.2
  vitest: latest => 0.24.4

Used Package Manager
npm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Update your Node version to latest Node 16.x at least

But the document says it requires node >= v14 to use vitest v0.24.4. I think one of them should be fixed.
By the way, I circumvented this issue by downgrading vitest to 0.24.3 :)

Yes, Vitest should support it. I just told how to fix it in the meantime 😄

I have the exact same problem with version 14.16.0.


Separate service LB & SD from network plumbing

EnableService and DisableService APIs allow a service to be accessible through Service Discovery and Load Balancer only if the image's healthcheck passes. Currently these APIs control the network plumbing as well. So if the healthcheck of an image tries to connect to other services on the same network it will not work. This PR separates the networking plumbing from the <Enable|Disable>Service APIs.
This will also help in the upgrade case where we want to take the service out of availability (ie: no new client connections to the service), but the existing connections should work till a grace period.
Signed-off-by: Santhosh Manohar<EMAIL_ADDRESS>

@aboch Addressed the comment. PTAL

LGTM

LGTM


nest error with snapshot_url

Version Information
(hass) root@cbus-dk-hass-01:/# hass --version
0.35.3
(hass) root@cbus-dk-hass-01:/# python3 --version
Python 3.5.2
Component/platform: Nest
Traceback (if applicable):

17-01-15 00:33:31 homeassistant.components.http: Serving /api/camera_proxy/camera.downstairs to <IP_ADDRESS> (auth: False)
17-01-15 00:33:31 aiohttp.server: Error handling request
Traceback (most recent call last):
  File "/srv/hass/config/deps/aiohttp/web_server.py", line 61, in handle_request
    resp = yield from self._handler(request)
  File "/srv/hass/config/deps/aiohttp/web.py", line 249, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 209, in coro
    res = yield from res
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 209, in coro
    res = yield from res
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/http/__init__.py", line 427, in handle
    result = yield from result
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 186, in get
    response = yield from self.handle(request, camera)
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 204, in handle
    image = yield from camera.async_camera_image()
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 91, in async_camera_image
    None, self.camera_image)
  File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/nest.py", line 98, in camera_image
    url = self.device.snapshot_url
  File "/srv/hass/config/deps/nest/nest.py", line 1238, in snapshot_url
    if self._device['snapshot_url'] != SIMULATOR_SNAPSHOT_URL:
KeyError: 'snapshot_url'
17-01-15 00:33:40 homeassistant.components.http: Serving /api/camera_proxy/camera.downstairs to <IP_ADDRESS> (auth: False)
17-01-15 00:33:40 aiohttp.server: Error handling request
Traceback (most recent call last):
  File "/srv/hass/config/deps/aiohttp/web_server.py", line 61, in handle_request
    resp = yield from self._handler(request)
  File "/srv/hass/config/deps/aiohttp/web.py", line 249, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 209, in coro
    res = yield from res
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 209, in coro
    res = yield from res
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/http/__init__.py", line 427, in handle
    result = yield from result
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 186, in get
    response = yield from self.handle(request, camera)
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 204, in handle
    image = yield from camera.async_camera_image()
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 91, in async_camera_image
    None, self.camera_image)
  File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/nest.py", line 98, in camera_image
    url = self.device.snapshot_url
  File "/srv/hass/config/deps/nest/nest.py", line 1238, in snapshot_url
    if self._device['snapshot_url'] != SIMULATOR_SNAPSHOT_URL:
KeyError: 'snapshot_url'
17-01-15 00:33:50 homeassistant.components.http: Serving /api/camera_proxy/camera.downstairs to <IP_ADDRESS> (auth: False)
17-01-15 00:33:50 aiohttp.server: Error handling request
Traceback (most recent call last):
  File "/srv/hass/config/deps/aiohttp/web_server.py", line 61, in handle_request
    resp = yield from self._handler(request)
  File "/srv/hass/config/deps/aiohttp/web.py", line 249, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 209, in coro
    res = yield from res
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 209, in coro
    res = yield from res
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/http/__init__.py", line 427, in handle
    result = yield from result
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 186, in get
    response = yield from self.handle(request, camera)
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 204, in handle
    image = yield from camera.async_camera_image()
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/__init__.py", line 91, in async_camera_image
    None, self.camera_image)
  File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/hass/lib/python3.5/site-packages/homeassistant/components/camera/nest.py", line 98, in camera_image
    url = self.device.snapshot_url
  File "/srv/hass/config/deps/nest/nest.py", line 1238, in snapshot_url
    if self._device['snapshot_url'] != SIMULATOR_SNAPSHOT_URL:
KeyError: 'snapshot_url'

Problem-relevant configuration.yaml entries and steps to reproduce:
nest:
   client_id: MYIDTOKNOWYOURSNOT
   client_secret: MYSECRETNOTYOURS
sensor:
  - platform: nest
    monitored_conditions:
      - 'temperature'
      - 'target'
      - 'humidity'

Additional info:

Seeing the same error with hass 0.38.3 ... the camera previews all show "Error loading image".

I figured it out ...
First, when creating the product (following the instructions here) on the Nest Developer account, I had selected just the "Camera" permission and not "Camera + images" which seems to be required for the snapshot_url call, according to the API docs.
Then, I had to log into my Nest account and "allow" the producto access the requested permissions. After that, the snapshot_url call started working and my camera images showed up in Home Assistant.

My Nest cameras have been working fine, thanks!


New components : list box and hierarchal list

Is your feature request related to a problem? Please describe.
We are from Assure Reinsurance team. While developing a Web application for existing desktop application we came across few components which are not available in CDK toolkit.

List box with multi selector
Hierarchical list

Existing designs in desktop application
List box

Hierarchical list and table

Describe the solution you'd like
For list box we have come up with the design based on CDK components.

Any update on this component development? As this component is used very often in our project and this doesn't have any alternative components.

Hi @Sr44102
Halstack Designers will discuss and propose an alternative Design to this use case.

More inputs shared by @Sr44102
Regarding the last pointer about styling the Typo:Case bases we need to change if settings in the design, it will be a huge constraint to just work with the default settings for fonts. I agree if we don’t add styling the code will be simple, but if we can have a option to create a style and that can be used across the project that would have been great.For example I want the font with bold, Purple, 14px.

<DxcTypography                 fontWeight="400" color="#5F249F"fontSize="0.875rem"  letterSpacing="0.025em" >Place your text here
Place your text here

 If I can create a style and that could be added for required Typography tag, that helps a lot in Keeping the code clearOne place declarationAny modifications will be easyStyle consistence across the project

Hi @Sr44102 , can you please share a link to your design so we can review it?

Hi @kukoc
The Figma link for the multi selector design : Multi Selector
It's not fully functional.

Hello @Sr44102 ,
We've reviewed the Figma file provided and have some comments regarding the design.
For the left container, we’d like to highlight some things:

We suggest using the scrollbar we currently have in our Dropdown list component

The icons to expand and collapse each dropdown list should be an “expand more” when the list is collapsed (the arrow pointing downwards) and an “expand less” (arrow pointing upwards) when the list is expanded. These arrows are being used in our Accordion component, please check this site for further reference.

In addition, regarding the buttons placed above these two containers, we suggest keeping just one primary button at the time depending on where the user is transferring from (left-to-right or right-to-left). This means:

When the user is selecting items on the left, the button that carries the action to move said items to the right should be primary, while the button placed on the left needs to be secondary.
However, if the user is selecting options on the left side, this button should be primary, and the other one secondary.
In short, the button associated witht the action that the user is performing is the one that should be primary, as having two primary actions one next to the other can be confusing for the user.

Another option that you can explore is having this type of layout with buttons separating both containers, like this:

I hope this can help your team, and don’t hesitate to reach out again if you have more questions.
Thank you!

Hi @Sr44102
Just to be clear, we haven't started designing any Halstack component for this use case. As iván mentioned, it's quite specific and we should review in detail why the UX can't be accomplished using Halstack components. If that's really the case, you can go ahead and move forward with the recommendations provided by Lara.
Please let us know, so we can close this issue in case we have an agreement. Thank you!

Hi @kukoc
I can make those changes to the design, but we need to implement the component on our own right? As we have this requirement in many screens.
Thanks @lcordobapuerto for the inputs, will get in touch in case of any clarifications.

Hi @Sr44102 . Since Cede is under a full UX redesign, does it make sense to keep this issue open ar should we close it?

Hi @kukoc
Sure. As we are exploring new designs for Cede, we can close this requirement.


Fix distribution creation

This PR contains the work that @Avalancs did in relation to issue #181 however due to changes that have since occurred in the codebase, it no longer functions fully :/
Current Status
Crashing
How to test

run gradlew build (this includes the key task distZip and is necessary to update the game binaries)
extract desktop/build/distributions/DestinationSol.zip
run sol.exe (or sol.sh)

Milestones

[x] Fix missing src/main/resources/icon.png crash
[ ] Fix ModuleManager.printAvailableModules NPE crash
[ ] Fix whatever else crops up crash
[ ] Do something with any TODO:s in the code

Hooray Jenkins reported success with all tests good!

Concerning the ModuleManager NPE: by adding a whole bunch of textfile.println()s (I'm almost certain there's a better way but ¯_(ツ)_/¯ ) I was able to identify that it fails at ModuleManager.loadEnvironment(...)
I also tried incorporating the changes from #259 but that didn't seem to help

Alright lots to sort through on this one :-)
IIRC the old goal was to simply let the zip work again. Yet since then we also have more module stuff going on. You think dealing with both at once is needed? It may well be.
In testing locally some notes:

Loads of debug logging indicating first search for engine assets happens under modules/engine - should that be a special case somewhere so the engine path is checked first?
On getting myself killed then respawning I hit an NPE at org.destinationsol.game.PlayerCreator.grantPlayerMoney(PlayerCreator.java:152) - didn't happen if I swapped back to develop although for some reason I respawned in the loaded imperial large while first I had picked the regular little wimpy imperial
Looks like a unit test or two may be problematic when run via Gradle. Seemed OK in IntelliJ. Log snippet below. Looks like the print thing already mentioned. Same thing that crashes when run as exe
A .tar file generates as part of the distribution plugin I figure (as part of gradlew build distZip but not from a gradlew clean test then gradlew distZip huh). Do we really need it? Seems like it double-includes something anyway, it is a bit more than twice the size of the zip

Log snippet for failing test:
org.destinationsol.game.ShardTest STANDARD_ERROR
    java.io.IOException: Could not resolve module metadata for module at C:\Dev\DestinationSol\Git\integrate\DestinationSol\engine\build\classes\main
        at org.terasology.module.ModuleFactory.createPathModule(ModuleFactory.java:189)
        at org.terasology.module.ModuleFactory.createModule(ModuleFactory.java:168)
        at org.destinationsol.ModuleManager.<init>(ModuleManager.java:52)
        at org.destinationsol.SolApplication.create(SolApplication.java:85)
        at com.badlogic.gdx.backends.lwjgl.LwjglApplication.mainLoop(LwjglApplication.java:143)
        at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:120)
    Exception in thread "LWJGL Application" java.lang.NullPointerException
        at org.destinationsol.ModuleManager.printAvailableModules(ModuleManager.java:84)
        at org.destinationsol.SolApplication.create(SolApplication.java:88)
        at com.badlogic.gdx.backends.lwjgl.LwjglApplication.mainLoop(LwjglApplication.java:143)
        at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:120)

More tinkering to come this is just my round one ...

Hooray Jenkins reported success with all tests good!

You get a NullPointerException when you run the application with the embedded jre but using the system jre, e.g. by running the command java -jar libs/solDesktop.jar, does not throw an exception. The game then starts correctly and appears to function with any further issues.
The crash has a stacktrace relating to a missing java.nio.file provider. The missing provider appears to be the Zip Filesystem Provider.
The bundled jre folders, which are used by the launch scripts (sol.exe, sol.sh, sol64.sh and solOSX.sh), are missing the provider jar file needed to run the application. The jar file is called zipfs.jar and by copying it from another jre distribution (i.e. the one installed locally on the machine) into the lwjre[OS]/lib/ext folder the game should run.
This may be resolved by #413, as the jre distribution will be removed in that pull request. I have obtained a copy of the zip provider jar from the com.github.marschall:zipfilesystem-standalone maven package, which I have added as a dependency of the engine project (since the engine uses gestalt-module). The package appears to be copy of the original source, abeit with a changed namespace. You can find the source here.
This pull request appears to run perfectly after the zipfs jar is added.
You can find the changes that I've made in my fork, on the fix-distribution branch. Should I make a pull request with these changes? The changes are alike to those provided by @Avalancs but they also add some code to package the modules with the distribution and do not utilise the gradle distribution plugin.
In terms of packaging the modules with the distribution, there appears to be some caveats. The LibGDX library only accepts FileHandle objects for assets, whilst Gestalt returns InputStream objects to access the file contents directly. To overcome this, the FileHandle class is sub-classed to provide access to a backing AssetDataFile, which is provided by Gestalt. This allows the files to be read from jar archives (zip archives do not work at the moment). The only asset type not using this method is the BitmapFont, which access the files directly and relies on a navigable file system being present.
Please let me know if there is a better solution to this, or if any more work needs to be done.
Note: this has not been tested on Android.

Hey @BenjaminAmos, nice to hear from you again! I'd say that if your code accomplishes something, open the PR. Someone who knows a bit more about our gradle stuff than me should look at it tho. One think I noticed from my quick scourge over your code is that unless there is something special happening for core, it should not have it's own build.gradle file in git, and all of the modules should have this file gitignored, even if only for security reasons (not implying that DestSol currently has any sort of security procedures in place :grin:). For the same reason on each ./gradlew build or wherever that piece of code is handled the build.gradle should probably be just deleted and recopied - but that's for another time, and probably not of any importance.

Superseded by #415 - closing as complete/merged via inspiration :-)


Add container's cgroup readonly mount documentation.

Signed-off-by: Mrunal Patel<EMAIL_ADDRESS>

See #62

The container should only have /sys/fs/cgroup if it mounts sysfs at
/sys.  That's the current default for runc:
$ runc spec
…
{
"type": "sysfs",
"source": "sysfs",
"destination": "/sys",
"options": "nosuid,noexec,nodev"
}
…
but I don't see sysfs mentioned in the specs yet.

I don't think I agree with this change. My understanding, from talking to many people, is that the only correct way to make this work is to bind mount the host cgroup hierarchy here and then optionally bind mount the container tree read-write.
The problem is that cgroups do not use relative paths in a number of places.
Also I am certain that we will want a read-write subtree to support use cases like pods. cc @vmarmol @rjnagal

you can have it RW as well and properly nest in the correct cgroup because it's a subtree for the container.

For context here is how we use the cgroup hierarchy in rkt today: http://www.spinics.net/lists/linux-containers/msg31037.html
This explanation provides concrete examples. So, could either of you (@mrunalp or @crosbymichael) contrast that with what you are proposing here?
cc @iaguis @alban

@philips
Here is what we are proposing (@crosbymichael can correct me).
We mount container's own cgroups at this path.
/ # ls -l /sys/fs/cgroup/
total 0
drwxr-xr-x    2 root     root             0 Jul  9 16:52 blkio
drwxr-xr-x    2 root     root             0 Jul  9 16:52 cpu,cpuacct
drwxr-xr-x    2 root     root             0 Jul  9 16:52 cpuset
drwxr-xr-x    2 root     root             0 Jul  9 16:52 devices
drwxr-xr-x    2 root     root             0 Jul  9 16:52 freezer
drwxr-xr-x    2 root     root             0 Jul  9 16:52 hugetlb
drwxr-xr-x    2 root     root             0 Jul  9 16:52 memory
drwxr-xr-x    2 root     root             0 Jul  9 16:41 name=systemd
drwxr-xr-x    2 root     root             0 Jul  9 16:52 net_cls,net_prio
drwxr-xr-x    2 root     root             0 Jul  9 16:52 perf_event

What do you mean with the container's own cgroups? If you mean that you mount a sub-tree of the cgroup, this will not work because /proc/$PID/cgroup lists full paths and the full paths would not match what is in /sys/fs/cgroup/. /proc/$PID/cgroup is unfortunately not namespace-aware (although there was some unfinished effort to make it more namespace-aware in the kernel).
Systemd requires the paths in /sys/fs/cgroup/ and /proc/$PID/cgroup to match. This is documented in the Systemd container interface (section "What You Shouldn't Do").

@mrunalp i would vote we don't have this as a default yet since there are so many ways of doing it.
@alban the paths for /proc/$PID/cgroup and /sys/fs/cgroup is a good argument for not doing this but I don't think much in that linked document apply because their recommendations are very insecure just make systemd work they way they want it to work.

@crosbymichael I agree.

@crosbymichael It seems like we should finish this discussion about sub-tree vs. non-subtree mapping and if we need to define an environment variable. I am fine saying the cgroup tree is RO, for now, but I would like to figure this tree stuff out here for @mrunalp's use case of a process wanting to know its own cgroups.

@mrunalp Maybe you could open an issue for discussion?

@philips I have opened https://github.com/opencontainers/specs/issues/66 to continue the discussion.


Tweak kubeadm release notes

Add some more detail and references.
This is a continuation of #211.
Astute readers will notice that I accidentally checked this in directly and then reverted it.  Sorry about that.

@lukemarsden @calebamiles @bgrant0607 @idvoretskyi

I didn't list every changed command-line flag as I think it would create more noise than be helpful.

Also /cc @luxas @pires @errordeveloper

Spot checking, this doesn't appear to address all the open review comments of #211. Did it intend to or were you just trying to move it closer?


Normalize metrics of text box when padding and fill modes change.

Summary
Normalizes the spacing of text boxes when fill mode or padding is changed.
This set ups the base for applying behind-the-scenes padding on text boxes when the highlight or fill mode is applied.
Relevant Technical Choices

To-do

User-facing changes

Testing Instructions

Create a new story or edit an existing story
Add some text boxes
Add a highlight or fill mode to the text box and see the text stays in place
Add some padding to the text box and see the text stays in place and the frame grows around it

Fixes #

@miina Correct. I'm splitting it up since I think there are still diverging ideas on how to approach the hidden text padding.

@miina Correct. I'm splitting it up since I think there are still diverging ideas on how to approach the hidden text padding.

since I think there are still diverging ideas on how to approach the hidden text padding.

I've done some exploration here, but curious to hear any and all approaches thought of thus far. Currently am growing the top level positioning container by the hidden padding amount, then insetting a deeper text child node by the hidden padding.
I think this approach logically makes sense with how it plays with our positioning system and the animation wrappers measuring relative to element. However, I don't love this solution since all resize measurements are taken from measuring the frames layer which would be off by the padding amount and need to be accounted for when reading the resized value. Also from a code perspective there doesn't seem like a centralized place to do this, seems like a bunch of small escape hatches scattered all over the place

since I think there are still diverging ideas on how to approach the hidden text padding.

I've done some exploration here, but curious to hear any and all approaches thought of thus far. Currently am growing the top level positioning container by the hidden padding amount, then insetting a deeper text child node by the hidden padding.
I think this approach logically makes sense with how it plays with our positioning system and the animation wrappers measuring relative to element. However, I don't love this solution since all resize measurements are taken from measuring the frames layer which would be off by the padding amount and need to be accounted for when reading the resized value. Also from a code perspective there doesn't seem like a centralized place to do this, seems like a bunch of small escape hatches scattered all over the place

Maybe a better approach for adding hidden padding is just adding the padding for the element on text bg mode selection and in the padding panel subtracting it out on display. Seems like it would make the change smaller, more centralized & not alter the existing positioning setup. any thoughts?

Maybe a better approach for adding hidden padding is just adding the padding for the element on text bg mode selection and in the padding panel subtracting it out on display. Seems like it would make the change smaller, more centralized & not alter the existing positioning setup. any thoughts?

Maybe a better approach for adding hidden padding is just adding the padding for the element on text bg mode selection and in the padding panel subtracting it out on display. Seems like it would make the change smaller, more centralized & not alter the existing positioning setup. any thoughts?

Seems reasonable to me, I was also initially thinking something similar. Adding padding to another wrapper could potentially get more complex than needed indeed.

Maybe a better approach for adding hidden padding is just adding the padding for the element on text bg mode selection and in the padding panel subtracting it out on display. Seems like it would make the change smaller, more centralized & not alter the existing positioning setup. any thoughts?

Seems reasonable to me, I was also initially thinking something similar. Adding padding to another wrapper could potentially get more complex than needed indeed.

Awesome. I'll explore that approach. Thanks for the feedback @miina !

Awesome. I'll explore that approach. Thanks for the feedback @miina !

Merged into https://github.com/google/web-stories-wp/pull/5839 instead.

Merged into https://github.com/google/web-stories-wp/pull/5839 instead.


Add setting to overwrite extension language

Hey there, first of all thanks for this extension!
I am using the extension in german Edge, and the german popups are... a bit weird. I would prefer to use the extension in english.
Might be a good addition to the settings menu.

I took a pass over the german translation and fixed a bunch of stuff.
This one was bugging me the most:

The german string was missing a space and the wording was absolutely butchered.
If you include this translation update I guess I would be fine using the german version, as the translation is mostly fine in the other places. Bit unlucky that the maybe most important place is so messed up currently 😅

A new release (1.18.5) with the updated translations has been submitted and will be available soon.

Works like a charm, thanks a lot <3

I edited the translation from 'spielt jetzt' to 'streamt jetzt', as the first one ('now playing') isn't working great for categories like just chatting. Don't stress merging it in, the current one is already great super useful.


Added python3 to cndefault

-Added python3 to cndefault.sh

python2 is now obsolete. Python (ie. python3) is expected to be available?
cndefault currently installs htop (is that required for all deployments?)
cndefault contains specific cyclecloud code (is that required for all deployments?)

if python2 is EOL I guess python3 will be added in images ?
I'm just cautious about the deployment time, htop is quick to install and is useful to troubleshooting, and CC code is only executed if needed. How much delay python3 installation will add here ?
if python3 is required for applications then it should be added as a deployment step and a dependency if needed.

ok, thanks will add it elsewhere. Please close this PR.


Checking  email exist on registration page via ajax

I try to convert signup pop with ajax but not getting desired outcome so can any suggest me solution for this

Please use the mailing list or StackOverflow for questions/help, where a wider community will be able to help you. We reserve the issues tracker for issues only.


Setting labels

hello
I was wondering whether it is possible to set some labels as well so I could have something like "All Books"?
Thanks

You can set the labels using the second parameter passed when creating a post type.
$books = new CPT('book', array(
    'labels' => array(
         'all_items' => 'All Books'
    )
));

For more label options see the WordPress Codex http://codex.wordpress.org/Function_Reference/register_post_type


Code navigation features

I have been using vim-javacomplete2 for some time, although not for large-project Java development.  My preferences for this plugin would be more basic code navigation features, as opposed to advanced refactoring or debugging capabilities.  I am already using exuberant tags along with the vim-easytags plugin, but this does not provide smart code navigation for Java.  I would think the Java parser used in javacomplete2 could support these features in a much better way.
In particular, the following features would be especially helpful.

Jump from a class/method/variable reference to its definition, taking into account method overloading (where there are multiple definitions for the same method with different signatures) and inner classes.
Display all the references for a given class/method/variable definition in the quicklist or location list, and use it to navigate to those locations.
Rename a method or variable across the project code base ('lightweight' refactoring).

If these feature are already available somewhere in javacomplete2 and I missed them, please let me know.

I use gtags in Windows 7. please search gun global as the key word.

Shidong Wang
-------- 原始邮件 --------
主题：Re: [artur-shaik/vim-javacomplete2] Code navigation features (#417)
发件人：Fred Yeadon
收件人：artur-shaik/vim-javacomplete2
抄送：Wang Shidong ,Comment
@artur-shaikhttps://github.com/artur-shaik I will try over the next few weeks. I am not at all familiar with gtags, but saw mention that it has not been implemented for Windows. Nevertheless, I will try to use it in either Linux or Windows and let you know.
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/artur-shaik/vim-javacomplete2/issues/417#issuecomment-459403353, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMiJkgN2HC0_oeyotdyBNjTON_Ark3Ilks5vIxX1gaJpZM4aXIsM.

@artur-shaik @wsdjeg  I have just tried gtags with vim, and it appears to satisfy the first 2 of the 3 requirements I mentioned (navigation but not renaming).  On Windows there were some problems with the backslash path separator, but they were not difficult to work around with vim mappings.
I haven't tried gtags enough to assess its Java parsing capability, in particular support for Java 8+ constructs.  GNU Global project development continues but does not seem very active, so I suspect that the Java support may not be very current.  I will post any key issues that I find.
Thanks for your quick replies to my original post.

@fwy you see, the third requirement you mentioned, can be related to #18 issue. It won't be realized earlier than I will proceed with those. So, you can close this issue, if others are suit you.

Closing this issue. I am seeing some problems with gtags Java parsing-navigation but may post those separately.


Use targetGradleVersion >=7.3 on new test

it shouldn't change behaviour from the perspective of an older tooling api version.

#20850

@bot-gradle help

Currently, the following commands are supported:
@bot-gradle test <BuildTrigger1> <BuildTrigger2> ... <BuildTriggerN>

A trigger is a special build for this PR on TeamCity, common triggers are:

SanityCheck/CompileAll/QuickFeedbackLinux/QuickFeedback/PullRequestFeedback/ReadyForNightly/ReadyForRelease
Shortcuts: SC/CA/QFL/QF/PRF/RFN/RFR
Specific builds:

BD: BuildDistributions/BuildDocs so that you can preview the generated docs/distribution.
PT: PerformanceTest, all performance tests for Ready For Nightly stage.
APT: AllPerformanceTest, all performance tests, including slow performance tests.
AST: AllSmokeTestsPullRequestFeedback
AFT: AllFunctionalTestsPullRequestFeedback
ASB: AllSpecificBuildsPullRequestFeedback
ACC: AllConfigCacheTestsPullRequestFeedback
ACT: AllCrossVersionTestsReadyForNightly
AFTN: AllFunctionalTestsReadyForNightly
ACTR: AllCrossVersionTestsReadyForRelease
AFTR: AllFunctionalTestsReadyForRelease

@bot-gradle test and merge

queues this PR for testing and merges if all tests pass by:

Creating a merge commit from your PR branch HEAD and the target branch
Running a ReadyForNightly build against the merge commit
When it passes, fast-forward the target branch to this merge commit (i.e. merge the PR)

The merge commit is called a pre-tested commit, which means that it fully tests the integration
of your branch HEAD and latest master, instead of only testing your branch HEAD.
@bot-gradle cancel

cancel a running pre-tested commit build or remove it from queue

@bot-gradle clean

clear the conversation history

@bot-gradle help

display this message

To run a command, simply submit a comment.  For detailed instructions see here.

@bot-gradle ACT

Sorry I don't understand what you said.
Currently, the following commands are supported:
@bot-gradle test <BuildTrigger1> <BuildTrigger2> ... <BuildTriggerN>

A trigger is a special build for this PR on TeamCity, common triggers are:

SanityCheck/CompileAll/QuickFeedbackLinux/QuickFeedback/PullRequestFeedback/ReadyForNightly/ReadyForRelease
Shortcuts: SC/CA/QFL/QF/PRF/RFN/RFR
Specific builds:

BD: BuildDistributions/BuildDocs so that you can preview the generated docs/distribution.
PT: PerformanceTest, all performance tests for Ready For Nightly stage.
APT: AllPerformanceTest, all performance tests, including slow performance tests.
AST: AllSmokeTestsPullRequestFeedback
AFT: AllFunctionalTestsPullRequestFeedback
ASB: AllSpecificBuildsPullRequestFeedback
ACC: AllConfigCacheTestsPullRequestFeedback
ACT: AllCrossVersionTestsReadyForNightly
AFTN: AllFunctionalTestsReadyForNightly
ACTR: AllCrossVersionTestsReadyForRelease
AFTR: AllFunctionalTestsReadyForRelease

@bot-gradle test and merge

queues this PR for testing and merges if all tests pass by:

Creating a merge commit from your PR branch HEAD and the target branch
Running a ReadyForNightly build against the merge commit
When it passes, fast-forward the target branch to this merge commit (i.e. merge the PR)

The merge commit is called a pre-tested commit, which means that it fully tests the integration
of your branch HEAD and latest master, instead of only testing your branch HEAD.
@bot-gradle cancel

cancel a running pre-tested commit build or remove it from queue

@bot-gradle clean

clear the conversation history

@bot-gradle help

display this message

To run a command, simply submit a comment.  For detailed instructions see here.

@bot-gradle test ACT

I've triggered the following builds for you:

AllCrossVersionTestsReadyForNightly build

@bot-gradle test ACTR

I've triggered the following builds for you:

AllCrossVersionTestsReadyForRelease build

@bot-gradle test this

I've triggered the following builds for you:

PullRequestFeedback build

@bot-gradle test ACTR

I've triggered the following builds for you:

AllCrossVersionTestsReadyForRelease build

@bot-gradle test and merge

I've triggered a build for you.


Scheduling of Surveys should take into account "locked_at" attribute on answer session

Followup answer sessions should only be created when the initial encounters answer session has been locked. AnswerSessions should get a new column called locked_at, and this should be used to compute whether a user should be assigned a new survey or not.
Example 1:
About Me Survey
Baseline (0days)
Followup (10days)
User signs up on August 1st.
User finishes baseline on August 2nd.
Followup reminder email should be sent on August 12th.
Example 2:
Survey After 12 days (no "baseline" encounter)
How have the first 12 days been? (12days) encounter
User signs up on August 2nd.
User gets "followup" reminder email sent on August 14th, even though no baseline (0day) answer session exists.
This should also take into account the "Publish Date" of the survey.
Example 3:
Survey After 5 Days (published on August 4th)
How have the first 5 days been? (5 days) encounter
User Signs up on August 2nd.
User gets followup email on August 9th (5 days after survey publish date)
cc @krand7

Encounters after baseline should only get assigned if baseline is complete, and all following encounters should happen relative to the baseline locked at date (regardless of interim answer session completion).

Checks are in the following order:

get earliest answer_session.created_at date
if it exists, base new encounter assignments off of this
if it does not exist

compare against survey.publish_date or user.created_at date and get offset instead.

Also, only assign ONE answer session. If no answer sessions have been completed, don't assign more.

So we're simplifying this a bit further. publish_date is being removed from surveys, and survey encounters will be dependent on other survey_encounters being completed (optional), here is the related issue: #807


[Auth] Do not permit to delete/edit Admin user, Admin group and permissions type system

Test OK.

Can't possible remove and edit admin group

Can't possible remove admin user

Admin user: email and name can be change

Can't delete a system permission


Swift Version not working

import UIKit
class ViewController: UIViewController, TTSlidingPagesDataSource, TTSlidingPageDelegate{
// var slider  = TTScrollSlidingPagesController()
var slider : TTScrollSlidingPagesController!

override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view, typically from a nib.
    self.slider = TTScrollSlidingPagesController()
    self.slider.titleScrollerInActiveTextColour = UIColor.grayColor()
    self.slider.titleScrollerBottomEdgeColour = UIColor.darkGrayColor()
    self.slider.titleScrollerBottomEdgeHeight = 2

//        self.slider.hideStatusBarWhenScrolling = true
//        self.slider.zoomOutAnimationDisabled = true
    self.slider.dataSource = self
    self.slider.delegate = self
    self.view.backgroundColor = UIColor.blackColor()
    self.slider.view.frame = self.view.frame
    self.view.addSubview(self.slider.view)
    self.addChildViewController(self.slider)
}

func numberOfPagesForSlidingPagesViewController(source: TTScrollSlidingPagesController!) -> Int32 {
    return 2
}

func pageForSlidingPagesViewController(source: TTScrollSlidingPagesController!, atIndex index: Int32) -> TTSlidingPage! {

    var vc : UIViewController!
    switch index {

    case 0 :

            vc = TabOneViewController()
        break
    case 1 :
            vc = TabTwoViewController()
        break
    default:
        break
    }

   return TTSlidingPage(contentViewController: vc!)

}

func titleForSlidingPagesViewController(source: TTScrollSlidingPagesController!, atIndex index: Int32) -> TTSlidingPageTitle! {
    var title =  TTSlidingPageTitle()
    switch index {
    case 0 :
        title = TTSlidingPageTitle(headerText: "TabOneVC")
        break
    case 1:
        title = TTSlidingPageTitle(headerText: "TabTwoVC")
        break

    default:
        print("index not 0 or 1")
        break

    }
    return title

}

func didScrollToViewAtIndex(index: UInt) {
    print("Did scroll to view at \(index)")
}

override func didReceiveMemoryWarning() {
    super.didReceiveMemoryWarning()
    // Dispose of any resources that can be recreated.
}

}
I have used the exact files of your TabOneViewController , And TabTwoViewController however if you run this app, the Content Flashes and goes away. Can you please check this code and let me know whats wrong. I am using "Your Demo Classes TabOne and TabTwo View Controllers"

My Bridging Header -
#import "TTScrollSlidingPagesController.h"
#import "TTSlidingPage.h"
#import "TTUIScrollViewSlidingPages.h"
#import "TTSlidingPageTitle.h"
#import "TabOneViewController.h"
#import "TabTwoViewController.h"
#import "TTUITableViewZoomController.h"

i got this working myself. Can you please suggest how would i take off the Zoom effect while i scroll. I want a regular transition without zoom like Groupon.

@vjujjavarapu I am having the exact same problem. Do you mind sharing how you fixed the issue? ViewControllers are not showing up.

Hi,
Can you specify which issue the zoom effect or getting it to work in swift?
On Tue, Mar 29, 2016 at 10:49 PM, Ken Chung<EMAIL_ADDRESS>wrote:

@vjujjavarapu https://github.com/vjujjavarapu I am having the exact
same problem. Do you mind sharing how you fixed the issue? ViewControllers
are not showing up.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/TomThorpe/UIScrollSlidingPages/issues/50#issuecomment-203213214

Hi,
I apologize. I have not read the full email. The solution was do not use
Tom's TabOneViewController oTabTwoViewController.
Create Subclasses of UITableViewControllers and Rest of the part should
work fine.
Create a bridging header without the TabOne and TabTwo.
On Tue, Mar 29, 2016 at 10:49 PM, Ken Chung<EMAIL_ADDRESS>wrote:

@vjujjavarapu https://github.com/vjujjavarapu I am having the exact
same problem. Do you mind sharing how you fixed the issue? ViewControllers
are not showing up.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/TomThorpe/UIScrollSlidingPages/issues/50#issuecomment-203213214