Added 'reparent_child' Function to Node Class.

This is a proposal for an reparent_child method addition to the Node class. This function provides a shorthand method for adding a child that may already have a parent node. This has helped me with reducing code repetition in cases where I want to move a child quickly to another node. An example of what would be done before this patch:
func set_up_player_camera_prepatch(player):
	var socket = player.get_node("./CameraSocket")
	var p = camera.get_parent()
	p.remove_child(camera)
	socket.add_child(camera)
	camera.translation = Vector3.ZERO
	camera.rotation = Quat.IDENTITY.get_euler()

Becomes the following code after patch:
func set_up_player_camera(player):
	var socket = player.get_node("./CameraSocket")
	socket.reparent_child(camera)
	camera.translation = Vector3.ZERO
	camera.rotation = Quat.IDENTITY.get_euler()

While the difference may be minor, I believe that it helps create more concise code that allows scripting files to be more concise when passing nodes to other parts of a tree. I believe that the reparent_child terminology also makes it clear that this node will become the new parent regardless of the presence of a parent node on a given child.
Preservation of Global Transform
For children that have compatible transforms, you can use this method to preserve your global transform using an optional boolean argument.
func set_up_player_camera(player):
	var socket = player.get_node("./CameraSocket")
	socket.reparent_child(camera, true)

I'm am not fully against the idea. But IMHO, this has little use cases and can be workarounded easily.
So I am not sure it is worth merging.

First of all this I don't think this should be named adopt_child, there's no need to further the "family" metaphor and go into the troublesome waters of adoption if we can avoid it. reparent_child would be technically more accurate, and thus a better name.
Then I also don't see much use case for this myself, but it's a relatively often requested feature. Yet, if this is implemented it should maybe go further and include an option to preserve the global transform as requested in #21781.

it should maybe go further and include an option to preserve the global transform as requested in #21781.

I think this would be confusing. As global_transform is meaningful only for CanvasItem and Spatial, it would make no sense to add it to the Node class. Also, this this really easy to workaround by registering the global_transform before and applying after the reparenting.

Also, this this really easy to workaround by registering the global_transform before and applying after the reparenting.

While that's true, the same applies to the whole reparenting process. If you need to do some bookkeeping for the transform, might as well do the reparenting manually IMO. So either we provide full convenience (which indeed means here that the reparent method would need to be virtual and the transform part handled in CanvasItem and Spatial), or we keep the current status quo.

"family" metaphor and go into the troublesome waters of adoption if we can avoid i

Are you being serious? This is what you thought of after looking at this PR?

 This is what you thought of after looking at this PR?

Please, those kind of interpellation are not constructive feedback. And look a little bit aggressive. :)
Also, I did not mention it but I agree "reparent" is a more adequate term.

Then I also don't see much use case for this myself, but it's a relatively often requested feature. Yet, if this is implemented it should maybe go further and include an option to preserve the global transform as requested in #21781.

Yes, this would be relatively easy to add. I will make the suggested changes.
While I agree it's really easy to work around this issue, I basically made this change for my own use because I was sick of the boilerplate when it came to quickly moving nodes to different parts of the tree. I've been using it for a while and I personally enjoy it, I was mostly curious how others felt about it.
I think a preserving global transform option would be an easy addition and would help to reduce even more boiler plate code.

I am against this. There is absolutely nothing wrong with writing more code in this case, it's not like you are writing this a hundred times in a project.

I have still gone ahead with the proposed changes to allow for preservation of global_transform settings. I've also changed the name of the function and fixed a bug.
I will keep this branch updated with refinements to the design as necessary. Truth be told, I don't mind continuing to cherrypick this change into my own projects for my own use. I simply find it decent utility that I decided might be nice to push upstream when I was asked about it personally.

In my experience, reparenting always look like a bad practice/design but sometimes may be inevitable, if you can provide examples where is hard to do a clean reparenting maybe you will be able to convince other devs.
Complex situations may involve physics, inputs, signals.
ps: in case something like this is added, I'm in favor of the reparent term too.

As for the name, I really don't mind changing the name to something that avoids stepping in any troublesome waters.

Don't even worry about that :), there is nothing wrong with using adopt_child in code, and it has nothing to do with adoption centers or families.  reparent does make more sense in your case, however (IMO)
I appreciate your PRs, especially the RichTextLabel. You don't deserve this kind of response at all, you did nothing wrong.

TL;DR I agree with @eon-s. Please convince us with use cases.

This is my own opinion based on my own experience with Godot. You're welcome to disagree.
I have never really run into a circumstance where I need to reparent in runtime. Fortunately, every time I thought I need to reparent, I could find a more straightforward solution that doesn't involve it. Reparenting is hacky and it exposes more undefined behaviors as some node's properties are derived from parent and you have to choose what to keep carefully. IMO reparenting is a hassle in itself and simply adding a method won't solve it.
For tools development, I guess you move nodes around more often, although I think it's not any more complex than remove_child and add_child. Those methods do more or less what you would expect and you don't have to worry about parent-derived properties.
About the syntax itself
parent.reparent_child(child, true)

It is a victim of boolean trap and would add more confusion when you come back to the code after a while without proper commenting, which negates the effort to keep the code concise.
From what I've gathered in the past years, Godot API design was always about verbosity and clarity. We'd rather have longer code than unclear code. (Although you might be able to find counter-examples, especially in GDScript built-in functions, it's often an older code and doesn't represent the design goal really well.)
About feature creep
If I search for feature proposal tag in the closed/unmerged PR, I'll see some of the PR that make me think "Ah. If this was merged it'd be really handy that time when I wanted to do blahblahblah..." but I was doing just fine without it.
Every piece of code we merge become a burden to maintainers, uninterested users, and even players if the binary is not optimized. We have to discuss extensively about whether it'd be useful in general.
About naming
It's not really the choice of Godot devs. They simply follow the trend. Some may find some terminology more uncomfortable than others, and avoiding them is a good idea.
"Assume positive intent." :)

There is absolutely nothing wrong with writing more code in this case, it's not like you are writing this a hundred times in a project.

The are LOTS of methods that you don't use hundred of times in a project, and yet, they are still there. If this functionality can be achieved relatively simply, I don't see why it shouldn't be in game. It's more useful than it seems. In almost every of my projects, I write a utility script, which contains, among others, the reparent method. It's really useful e.g. when you attach a level scene, but want the objects to be inside game state (because YSort). Or want to e.g. move items between containers in Node-based inventory. Or want an object to stick to another i.e. become it's child upon collision. There are many examples for use of reparenting.

I think this would be confusing.

Confusing? You know what is confusing? Local position. Questions like "my node disappeared" are asked quite frequently. People manage to change the parent of a node, but have no idea that they have to keep the old global position. And personally, every time I have to do this it's just awkward, because you need to introduce a temporary variable (reparent's implementation would probably do the same, but whatever). If anything, the current "reparenting" is confusing.
When I created the issue linked by akien-mga, it was not because I randomly thought that since I just wrote it, it should be part of the engine. I created the issue, because I had to implement the method myself numerous times and there are lots of people who don't even know how to do it (let's ignore that they should just learn how to do anything). I bet many people will agree.

This has received a lot more traffic than I thought it would, so let me start from the top.
@eon-s

In my experience, reparenting always look like a bad practice/design

I understand that there are some cases where reparenting may not be the best solution, but it certainly isn't a bad practice in its own right. In fact, I believe that Godot is designed in a way where there's not really a clear 'one size fits all' solution that should be considered a go-to for every project. I believe that it really depends on how you are using node data and what your plans are for your game/app structure. For instance, node reparenting is absolutely a must in cases where you wish to recycle scene instances (instead of instantiating a scene multiple times) and can also be very handy when wanting to make use of local coordinates. Specifically, my current use case uses  'socket' nodes which allow you to attach meshes to one another, or also construct modular structures which are designed to socket into one another. I also use these sockets to quickly move the camera between desired targets (with interpolation being controlled by a camera script.) I also think it is useful in cases where you have 'spawner' entities that are responsible for spawning scenes. Also...
@avencherus

It comes up often enough for me. I write a lot of editor tools for my projects, and the code for moving things around would tighten up a bit, and be more readable if there were a reparent() method.

This is exactly how I feel as well. While it may be something that may not happen often in an actual runtime environment, it does come up pretty often when you are trying to make an editor tool that manages node data behind the scenes.

It is useful, and not project specific. Given the rest the functions on the Node it seems like it should've already been included from the start.

I agree -- however...

I would agree it isn't often enough that its breaking my wrist to write the extra lines, but at the same time I don't understand the resistance on the grounds of preventing feature creep? Is that the reasoning? I don't think these are the kinds of small additions that have to be guarded against.

To play devils advocate a bit, I actually understand the logic behind being sceptical toward a change to the api of a low level base class. Simply by adding a preserve_global_transform flag, I've already needed to provide custom behaviour in the case that users are using Spatial and Node2D derived classed. However, this adds to burden of maintenance (these functions will have to be updated whenever a major change happens to transform data) and also adds complexity to all of the classes that derive from these classes. And then there's going to be the inevitable situation where people want more data to be inherited from the parent (CanvasItem bounds, Node Visibility, Whatever Else) that would extend the complexity across (basically) every class.
So the problem isn't so much feature creep as much as it is resistance toward maintaining another function that does something similar but with a different implementation. Having said that, I feel that with the proper scope, a simple utility method for removing-and-adding a child would be easily maintainable since most of the actual code surrounding it is simply error handling.
@Naryosha

It is a victim of boolean trap and would add more confusion when you come back to the code after a while without proper commenting, which negates the effort to keep the code concise.

Yes, but frankly that hasn't actually stopped any other part of the code from using booleans as optional flags. So if we're to make a rule about this, it should be consistent across the entire API.

Reparenting is hacky and it exposes more undefined behaviors as some node's properties are derived from parent and you have to choose what to keep carefully.

Honestly, if done well, you can avoid these undefined behaviors by using _notification to handle these types of use cases. NOTIFICATION_PARENTED and NOTIFICATION_UNPARENTED are good for these specific scenarios and can help resolve any undefined behaviour. The better argument against node reparenting is that it makes it basically impossible to use NodePath to navigate to a desired node if it is moving about. However, that's very specific to the design of the application in question. I would never reparent a node that I was planning to use NodePath to access, with or without this function.
Lastly...

You don't deserve that kind of response at all, you did nothing wrong.

I feel the need to point out that people shouldn't really be taking discussions around any of this personally or think of it as such. I believe it's healthy for open source project that PRs are talked about, debated, and have all opinions shared. I'm actually very happy to have a discussion around this, and I actually went into this PR knowing full well that this would be a topic that needs heavy discussion. I simply wanted to share my own private solution and hear some opinions on why this isn't already in the API.

Would implementing this as a property (as described in #24433) be a better alternative for people? Some of the 'smart' behaviour introduced later in this patch would probably have to be changed or removed, but I may make for an even cleaner to be able to do something like:
some_node.parent = new_parent

and would also allow for exposure to signals / slots so that you could have your own custom parent-reassign behavior within a given node?

I'm going to go ahead and close this PR because of the new Godot Proposals ( godotengine/godot-proposals ) repository. Those you feel strongly about this feature (maybe @PLyczkowski @KoWeBi @avencherus and others?) should feel free to open an issue there and link to this PR for previous discussion.
I'm not sure how the migration from this board to the other is supposed to work, but closing this PR now is probably for the better and can be resubmitted at any time.


introduce -e cli flag for specfying the engine path in case of special characters

Codecov Report

Merging #85 (5d4a2de) into master (3517015) will decrease coverage by 2.71%.
The diff coverage is 65.78%.

@@            Coverage Diff             @@
##           master      #85      +/-   ##
==========================================
- Coverage   77.88%   75.17%   -2.72%
==========================================
  Files           5        5
  Lines         407      427      +20
==========================================
+ Hits          317      321       +4
- Misses         60       76      +16
  Partials       30       30

Impacted Files
Coverage Δ

pkg/printer/printer.go
81.42% <60.00%> (-2.68%)
:arrow_down:

cmd/root.go
68.00% <65.00%> (-7.56%)
:arrow_down:

pkg/utils/utils.go
74.50% <100.00%> (+1.59%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add instructions to restart and update boringproxy server and client

https://forum.indiebits.io/t/readme-outdated-pull-request-inkscape-command-issue/406

Is this repo still active?

@spenceryonce
No, I haven't looked at it in 5 months, but the updates should still be accurate


file-storage: List or remove multiple files with a common prefix

Once you accept multiple files at the same time as a draft, you can't clean up them without a database support in a draft preview page. You may want to clean up all files in stale drafts, too.
Common file systems can list or remove all files in a common group by ls -R or rm -rf. Boto3 can list_object with a prefix up to 1000 at once. However, the current file-storage can't do it. You have to find out every name of the file you want to delete first before actually doing it.

I've considered adding a keys() method to the FileStorage interface. This would allow you to scan through all keys and find the ones you want to delete.

For my R2 implementation of FileStorage and ended up extending it to add this keys method https://github.com/edgefirst-dev/kit/blob/44ee2a128e029bf7a5f0aad52cca47f696792a44/src/lib/fs/fs.ts#L61-L66
I also thought that instead of returning that promise, it could return an AsyncIterator, I ended up using a Promise instead because usually you just get one page at the time, and not the whole list.

AsyncIterator<string[]>?
I think (almost) all types of storages can fetch the list of several tens, hundreds, or thousands of files at once?
You can use the flatMap or convert it to an AsyncIterator<string>.

AsyncIterator<string[]> seems like a natural interface for paging through keys, yes. I'd be open to a PR that adds this functionality.

You need to return not only the keys, but a cursor so let you start pagination from where you left
Unless you're showing all keys at once to users you don't want to list all of them at once.

Would this API be sufficient, @sergiodxa ?
interface FileStorage {
  keys(cursor?: string): Promise<{ cursor?: string; keys: string[] }>
}

Yes, that's similar to what I did myself, I also sent a done boolean so you know there are not more things to fetch, which could be implicit if cursor is not sent too.

I wonder what type the cursor should be in storages whose structures are completely different from S3. Could it be number, bigint, or structure types, or should its type be a type parameter? The exposure of the cursor is not necessary for users, especially those of MemoryStorage and LocalFileStorage.
You will be able to use map or something similar with AsyncIterator in the future:
https://github.com/tc39/proposal-async-iterator-helpers
Lazy users can use await Array.fromAsync for AsyncIterator even now.
The suggested API forces users to use the procedural manner (the while statement).
Do MemoryStorage and LocalFileStorage prefer (Async)Iterator or AsyncIterator<Iterator> the most?

I wonder what type the cursor should be ... should its type be a type parameter?

Probably. I'd imagine string would be the default though.
interface FileStorage {
  keys<T = string>(cursor?: T): Promise<{ cursor?: T; keys: string[] }>
}

The exposure of the cursor is not necessary for users

I disagree. Imagine showing a web page with one page of keys, and a "Next page" button that will show the next page. In that case, users will need to have a cursor they can use to pass back in on the request for the next page of keys.

Imagine showing a web page with one page of keys, and a "Next page" button that will show the next page.

I see. I think we might as well provide both kinds.

If we want to keep the AsyncIterator interface there's an option to add a Symbol.asyncIterator property, that way the instance itself would be iterable.
let fs = new MyFileStorage()
for await (let key of fs) {
  // use key here
}

So if you want to iterate the whole list you can use that instead, and if you need a paginated list of keys you can use FileStorage#keys.

there's an option to add a Symbol.asyncIterator property, that way the instance itself would be iterable.

There are some following cons, so it is not a good idea to force implementers to implement it, or to serve only the property without an additional equivalent function at least.

cannot pass any additional options, e.g. prefix and # of entries in one fetch
induces concrete FileStorage implementers to make their FileStorage itself a state instead of returning a newly created async generator

AsyncIterator → .iterateKeys()
Object with cursor → .partialKeys() or some better names that can reduce misunderstanding (this function could never be paused) by users
AsyncIterator seems to be able to be calculated by objects with cursor, so now I think implementers have only to implement the function returning one with cursor.


Lifetime elision for impl

In similar style as fns unspecified lifetime parameters should all bind to a hidden parameter.

This was already accepted in RFC 411 https://github.com/rust-lang/rfcs/blob/master/text/0141-lifetime-elision.md
Nobody has implemented it yet. It's being tracked here https://github.com/rust-lang/rust/issues/15872


bug-fix - adding 'input' event to handler managing 'keyup' event to e…

bug-fix - adding 'input' event to handler managing 'keyup' event to ensure copy/paste submission to the control is read properly

putting the test failures on hold briefly, can you describe in a little more detail what problem this change resolves?
in Chrome 65, when i copy/paste the string "LAX" into this sample and click Enter, three results are displayed.

hmmm.... that's definitely not the behavior we were observing.  After pasting and hitting a carriage return, the control wasn't responding.  Perhaps Chrome 65 had an update?  It's hard for me to go backwards and test on Chrome 64 because we're on government systems (I know the update happened for us from 64 to 65 fairly recently).

One thing to note - we are loading a number of searches into this control... including searches against some of our own hosted map services.  Perhaps it behaves differently vs. these as opposed to searches vs. ArcGIS Online?

i'm confused. are you still able to reproduce the error or not?

Good question... let me check...

Ok, I removed the update and it's definitely working vs. Chrome 65.  I know this wasn't my imagination though as this issue had been replicated many times across multiple machines...  All I can suggest is perhaps Chrome made some kind of change that solved this problem for us shrug.  It's a good problem to have I guess - but I'd love to know what changed.

I'd love to know what changed.

me too 😺. for the time being lets just call it a (resolved) Chrome bug that we no longer need to workaround. if you're ever able to reproduce again in the future, i'm happy to revisit.


perf: Cache subschemas

Continuation of #894. This patch introduces a cache for subschemas-related keywords, which reduces the number of calls to the search routine. All subschemas-related keywords are searched in one pass and then stored in a cached dict. With the OP from #893 it reduces the total number of function calls from 61512655 to 56438634 (~9%). The number of calls to the search routine is reduced 5x (from 1090000 to 218000).
I'd be happy to know what do you think :)

Wonderful, thanks again, merging!


#488 - Cannot update labels if entity is loaded in a new session

Description

Merging primaryId mapping for both nodes and relationship
Remembering identity with existing ID without calling nativeId method
Reusing identity update logic for both nodes and relationships
Creating tests in EntityAbstractionTest and PizzaIntegrationTest
Testing label updates with graphId and different primaryId using @Id

NOTE:

Those changes depend on #485. Please, do not review it until #485 is approved and merged.

Related Issue
#488 - Cannot update labels if entity using @Id is loaded in a different session
Motivation and Context
Please, read #488
How Has This Been Tested?
Using existing test case in addition to new tests
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

I have incorporated your tests. They are all green now. The solution looks a bit different, though.
I like your approach but as we have some other plans and also some work going on (you might want to have a look at this branch it's hardly mergeable.
The simpler solution was either fixing the order of when to access nativeId or just use the id that we already have during mapping.
Thanks again for your contribution.


 When preforming streaming operations we don't need any version of the table loaded. Large tables are slow to load, so we see a huge performance boost by avoiding CPU time spent loading the table metadata.
Related Issue(s)
Partly with https://github.com/delta-io/delta-rs/issues/1361
Testing
Adding Unit tests
Breaking Change
Not a breaking change.

@PengLiVectra - could you explain the use case a bit?
For typical scenarios, without protocol and metadata information there is not much we can (legally) do with the table.
To avoid loading the state into memory we can already pass options to omit files and tombstones. Currently we are also working on making that more lazy - i.e. just getting protocol and metadata asap, but avoid loading anything else.

@PengLiVectra - could you explain the use case a bit?
For typical scenarios, without protocol and metadata information there is not much we can (legally) do with the table.
To avoid loading the state into memory we can already pass options to omit files and tombstones. Currently we are also working on making that more lazy - i.e. just getting protocol and metadata asap, but avoid loading anything else.

Yes, when using in streaming operations, sometimes the task has no data, it takes too much time to load large tables. We did a simple test with load_lazy:
cat test.py
import deltalake x = deltalake.DeltaTable(s3path, lazy_load=True) print(x.peek_next_commit(4690))
If load_lazy = True, result is python test.py  0.89s user 0.39s system 84% cpu 1.526 total
If load_lazy = False, result is python test.py  97.53s user 3.74s system 87% cpu 1:56.23 total

@PengLiVectra if the task has no data, and you can check this why not simply load the table state if there is data, if not you skip the step.

I still don't quite understand the use case :)
If your task receives no data, then there should be no need to lad the table at all, right? But when does hte need to peek the next table version arise?
delta-rs was initially build to support large scale log ingestion into delta via kafka streams (this is also why the kafka-delta-ingest project exists)
To handle the sometimes very large state that gets generated there are two options that can be passed to the table require_files and require_tombstones which will cause the internals to not load the file actions (we don't want to do a scan) and the tombstones (we also don't want to vacuum).
for any other meaningful interaction with the table, we always need the protocol and metadata actions. for the above scenario it would also need to be an appendOnly table.
So from the litle I know about your use case, I would suggest:

configure table as append only.
check you data if you need to write to the table at all
if you need to write  data, load the table with require_files and require_tombstones set to false?

Would that meet your needs?

3. require_files

Sorry for the misleading @ion-elgreco and @roeap , we didn't know if there is no data prior, we need to load the table and check if there is a new commit. So basically we need to save the time in the step of loading table.

@PengLiVectra also in that case, wouldn't initializing the table without files give the same result combined with get_latest_version or peek next commit

@PengLiVectra if the task has no data, and you can check this why not simply load the table state if there is data, if not you skip the step.

@PengLiVectra also in that case, wouldn't initializing the table without files give the same result combined with get_latest_version or peek next commit

@PengLiVectra also in that case, wouldn't initializing the table without files give the same result combined with get_latest_version or peek next commit

Thanks, I'm still not very clear. Could you give an example about it in Python?

I still don't quite understand the use case :)
If your task receives no data, then there should be no need to lad the table at all, right? But when does hte need to peek the next table version arise?
delta-rs was initially build to support large scale log ingestion into delta via kafka streams (this is also why the kafka-delta-ingest project exists)
To handle the sometimes very large state that gets generated there are two options that can be passed to the table require_files and require_tombstones which will cause the internals to not load the file actions (we don't want to do a scan) and the tombstones (we also don't want to vacuum).
for any other meaningful interaction with the table, we always need the protocol and metadata actions. for the above scenario it would also need to be an appendOnly table.
So from the litle I know about your use case, I would suggest:

configure table as append only.
check you data if you need to write to the table at all
if you need to write  data, load the table with require_files and require_tombstones set to false?

Would that meet your needs?

require_files and require_tombstones are in build function, not in load function
https://github.com/delta-io/delta-rs/blob/main/crates/deltalake-core/src/table/builder.rs#L280-L300.
It seems only load functions is exposed in python (https://github.com/delta-io/delta-rs/blob/main/python/src/lib.rs#L116).
Is there a way to use this in Python?

@PengLiVectra that's likely because it doesn't load any state and also doesn't validates the file path unlike the normal load would do
@roeap please correct me if I'm wrong

@PengLiVectra - sorry for the large delay.
We are currently working towards release 0.17 which will include many breaking changes. One large piece of work I am currently addressing is to optimize the whole log handling and be as minimal as possible in getting to a useful state of the delta table. We are also leaking a lot of internals that we cannot commit to long term.
Why am I telling you this? :)

TL;DR; would it be OK to wait with a way forward a bit until some internal changes have landed?

Your use case makes a lot of sense and we certainly have the opportunity to provide an implementation that streaming use cases can benefit from - much like you proposed.
The lazy-load in this case is more on the object store level, since if we do not have any metadata and without it we cannot do anything meaningful specific to delta. In fact the fact that the metadata and protool are currently optional is a major nuicance, that permeates the entire codebase ... IIRC, you are most interested if a new version of the table exists, right?
In that case we will also have facilities to inspect the state of the log from a storage perspective. and we may expose some of this for use cases like yours.

@PengLiVectra - sorry for the large delay.
We are currently working towards release 0.17 which will include many breaking changes. One large piece of work I am currently addressing is to optimize the whole log handling and be as minimal as possible in getting to a useful state of the delta table. We are also leaking a lot of internals that we cannot commit to long term.
Why am I telling you this? :)

TL;DR; would it be OK to wait with a way forward a bit until some internal changes have landed?

Your use case makes a lot of sense and we certainly have the opportunity to provide an implementation that streaming use cases can benefit from - much like you proposed.
The lazy-load in this case is more on the object store level, since if we do not have any metadata and without it we cannot do anything meaningful specific to delta. In fact the fact that the metadata and protool are currently optional is a major nuicance, that permeates the entire codebase ... IIRC, you are most interested if a new version of the table exists, right?
In that case we will also have facilities to inspect the state of the log from a storage perspective. and we may expose some of this for use cases like yours.

Thanks, @roeap. Do you have a timeline for release 0.17?

@PengLiVectra now that we already using the new log replay in python as well, I believe this can be closed. Does it meet your needs?


replace some api to function

due to exception handling.
the following api is target.

printStackTop
reportError

also replace D-Bus event loop api

replace printStackTop to print expression in commit db591797f2f6e1095479a846153e9da4932887f0

replace reportError in commit 0bb3a819b8d8dc19dd3eba25358d501a1837a2d6

fix D-Bus wait signal method in commit cabe792364563aed9a5c8037c45020f8e21af573


Update Google OAuth adapter to move away from deprecated "Google+" APIs

Summary: Fixes T13227.
Test Plan: {F6154706}
Reviewers: amckinley, dragoon20
Reviewed By: dragoon20
Subscribers: dragoon20
Maniphest Tasks: T13227
Differential Revision: https://secure.phabricator.com/D20030
(cherry picked from commit 9456d6683d3388fbfe23503ff9f676a9186952e4)

This is in place on https://phabricator.dev.endlessm.com and it seems to work.


Site tree not updating titles

Hello, I have one problem with this otherwise great module. When locale is changed in the back-end, page titles in the Tree View don't update. This would be OK (they never update in Multilingual module, for instance) but if I reload the admin page, they do update to current locale. Moreover, the title of a saved page also updates, which results in the Tree View showing titles in mixed languages. Do I make any sense?
What I mean is, for example:

Locale is Polish. Tree View shows pages in Polish. OK.
Switch to English. Tree View still shows pages in Polish, although upon clicking them I can properly edit English data.
Edit some page and save it. Now it updates and shows in the Tree View in English, while other pages in the tree still show up with Polish titles. Confusing.
Reloading with CTRL-R fixes everything - whole Tree View shows titles in current locale.

Is this the intended behavior or have I done something wrong? Editing is fine and it's only a display issue with the Tree View, but it's very confusing for the editor.

Is there a fix for this in progress? Would be cool to have this in the official repo instead of local fixes.

The module used to just refresh the whole page (making everything refreshed) but it was requested that the behaviour is changed to use the standard soft ajax refresh, so it'll need to be fixed in core if we want to keep this.

Yep, this is something really important to be fixed or at least controlled via config.yml  so that we could easily choose - AJAX update or Reload. We are all awaiting :) Thanks

Yeah, I'll make it an option... or would someone like to make a pull request in the mean time? (I'm likely not to get around to this any time soon).

Here's the commit that made the loading via ajax... the change would partially be a reversion of the existing code, plus an additional option to switch between the old / new functionality.
https://github.com/tractorcow/silverstripe-fluent/commit/77d98e0f489db8ec74ad2d091b3b292c4bfb7d1a


Add out-of-the-box support for browser using Fetch API

Hope this is still being maintained, I have fork on my profile if someone needs.

this works great, would be cool to see it merged. either way, thanks @eioo!
worth noting, this is much better than the originally proposed "just replace readFile" as this also handles loading external tilesets nicely.


Use DerivesFromOrEqual instead of DerivesFrom for Context in resolve-during-install warning checks

Thank you for sending a pull request! Please make sure you read the contribution guidelines
Pull request checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been reviewed and added / updated if needed (for bug fixes / features)
[x] No new compiler errors or warnings

Pull request type

Please check the type of change your PR introduces:

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

Issue Number

Issue Number: #194
Create or search an issue here: Extenject/Issues
What is the current behavior?

Under certain circumstances, using a method that uses Container.Resolve<Context>() (e.g. FromComponentOnRoot) will cause an invalid warning. The following test demonstrates the issue:
public class ResolveWarningTest : ZenjectIntegrationTestFixture
{
	class TestFacade
	{
	}

	class TestInstaller : Installer<TestInstaller>
	{
		[Inject] private Transform transform;

		public override void InstallBindings()
		{
		}
	}

	[Test]
	public void FromComponentOnRoot_CausesWarning()
	{
		this.PreInstall();
		this.Container
			.Bind<TestFacade>()
			.FromSubContainerResolve()
			.ByNewGameObjectMethod(subContainer =>
			{
				subContainer.Bind<TestFacade>().AsSingle();
				subContainer.Bind<Transform>().FromComponentOnRoot();
				TestInstaller.Install(subContainer);
			})
			.AsSingle()
			.NonLazy();
		this.PostInstall();
	}
}

What is the new behavior?

No warning is produced in scenarios like this

Does this introduce a breaking change?

[ ] Yes
[x] No

Other information

Since this only affects warning output, I didn't do any extensive testing on this (should be safe)
Should a test like this be added to the test suite? Looking at the contribution guidelines, I feel like it's not in line with the general testing principles, and I couldn't easily find a category for it.

On which Unity version has this been tested?

[ ] 2020.4 LTS
[ ] 2020.3
[ ] 2020.2
[ ] 2020.1
[x] 2019.4 LTS
[ ] 2019.3
[ ] 2019.2
[ ] 2019.1
[ ] 2018.4 LTS

Scripting backend:

[x] Mono
[ ] IL2CPP

Note: Every pull request is tested on the Continuous Integration (CI) system to confirm that it works in Unity.
Ideally, the pull request will pass ("be green"). This means that all tests pass and there are no errors. However, it is not uncommon for the CI infrastructure itself to fail on specific platforms or for so-called "flaky" tests to fail ("be red").  Each CI failure must be manually inspected to determine the cause.
CI starts automatically when you open a pull request, but only Releasers/Collaborators can restart a CI run. If you believe CI is giving a false negative, ask a Releaser to restart the tests.

Thanks @sbergen
Merged!


kots CLI: "kubectl kots pull" creates invalid k8s objects

tl;dr kubectl kots pull seems pretty broken right now. This issue documents the problems and some manual workarounds to get around them.

I tried a kubectl kots pull using the published sentry example license and was unable to apply the resulting yaml. I've tried this with a few apps and it should be pretty easy to reproduce. There are a few issues here:

There's a big config map that's too big for etcd, we need to remove this
all resources in upstream/admin-console have a hardcoded namespace: default, regardless of what namespace is passed to kubectl kots pull
AUTO_CREATE_CLUSTER_TOKEN is not set by kots pull

There's one thing that I think is maybe an enhancement opportunity rather than a bug, but at the end of the deploy, kotsadm still wants you to upload a license, config, preflight checks, etc.
Repro steps
$ kubectl kots pull sentry-pro --license-file ~/go/src/github.com/replicatedhq/kots-sentry/KOTS-license-example-sentry-pro.yaml

Enter a new password to be used for the Admin Console: ••••••••
  • Pulling upstream ✓
  • Creating base ✓
  • Creating midstream ✓

    Kubernetes application files created in /Users/dex/sentry-enterprise

    To deploy, run kubectl apply -k /Users/dex/sentry-enterprise/overlays/midstream

But running that gives an error of The ConfigMap "kotsadm-bundle-0" is invalid: metadata.annotations: Too long: must have at most 262144 characters
$  kubectl apply --namespace sentry-pro -k /Users/dex/sentry-enterprise/overlays/midstream
I0725 10:41:54.540984   73897 log.go:172] nil value at `volumes.configMap.name` ignored in mutation attempt
I0725 10:41:54.541068   73897 log.go:172] nil value at `volumes.projected.sources.configMap.name` ignored in mutation attempt
I0725 10:41:54.541090   73897 log.go:172] nil value at `volumes.secret.secretName` ignored in mutation attempt
I0725 10:41:54.541178   73897 log.go:172] nil value at `volumes.projected.sources.secret.name` ignored in mutation attempt
I0725 10:41:54.541213   73897 log.go:172] nil value at `volumes.persistentVolumeClaim.claimName` ignored in mutation attempt
serviceaccount/kotsadm-api configured
serviceaccount/kotsadm-operator configured
serviceaccount/kotsadm configured
role.rbac.authorization.k8s.io/kotsadm-api-role configured
role.rbac.authorization.k8s.io/kotsadm-operator-role configured
clusterrole.rbac.authorization.k8s.io/kotsadm-role configured
rolebinding.rbac.authorization.k8s.io/kotsadm-api-rolebinding configured
rolebinding.rbac.authorization.k8s.io/kotsadm-operator-rolebinding configured
clusterrolebinding.rbac.authorization.k8s.io/kotsadm-rolebinding configured
configmap/sentry unchanged
secret/kotsadm-cluster-token configured
secret/kotsadm-encryption configured
secret/kotsadm-minio configured
secret/kotsadm-password configured
secret/kotsadm-postgres configured
secret/kotsadm-session configured
secret/sentry-postgresql configured
secret/sentry-redis unchanged
secret/sentry configured
service/kotsadm-api-node configured
service/kotsadm-minio configured
service/kotsadm-postgres configured
service/kotsadm configured
service/sentry-postgresql unchanged
service/sentry-redis-master unchanged
service/sentry-redis-slave unchanged
service/sentry unchanged
deployment.apps/kotsadm-api configured
deployment.apps/kotsadm-operator configured
deployment.apps/kotsadm configured
deployment.apps/sentry-cron configured
deployment.apps/sentry-postgresql configured
deployment.apps/sentry-redis-slave unchanged
deployment.apps/sentry-web configured
deployment.apps/sentry-worker configured
statefulset.apps/kotsadm-minio configured
statefulset.apps/kotsadm-postgres configured
statefulset.apps/sentry-redis-master configured
job.batch/sentry-db-init configured
job.batch/sentry-user-create configured
ingress.extensions/sentry-ingress unchanged
persistentvolumeclaim/sentry-postgresql unchanged
persistentvolumeclaim/sentry unchanged
pod/kotsadm-migrations-1595691679 created
The ConfigMap "kotsadm-bundle-0" is invalid: metadata.annotations: Too long: must have at most 262144 characters

Workaround step 1: removing config map
It seems this can be worked around by commenting the config map out of base/kustomization.yaml, but I am unclear as to whether this will break anything
# ~/sentry-enterprise/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- admin-console/api-deployment.yaml
- admin-console/api-role.yaml
- admin-console/api-rolebinding.yaml
- admin-console/api-service.yaml
- admin-console/api-serviceaccount.yaml
# - admin-console/kotsadm-bundle-0.yaml
- admin-console/kotsadm-deployment.yaml
- admin-console/kotsadm-role.yaml
# ... etc etc

Workaround step 2: overriding default namespace in kustomize
After removing the config map and doing another apply, we get a whole bunch of issues with hardcoded namespaces:
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.

$ kubectl create namespace sentry-pro
namespace/sentry-pro created
$ kubectl apply --namespace sentry-pro -k ~/sentry-enterprise/overlays/midstream
I0725 10:56:36.489386   80758 log.go:172] nil value at `volumes.configMap.name` ignored in mutation attempt
I0725 10:56:36.489470   80758 log.go:172] nil value at `volumes.projected.sources.configMap.name` ignored in mutation attempt
I0725 10:56:36.489503   80758 log.go:172] nil value at `volumes.secret.secretName` ignored in mutation attempt
I0725 10:56:36.489630   80758 log.go:172] nil value at `volumes.projected.sources.secret.name` ignored in mutation attempt
I0725 10:56:36.489669   80758 log.go:172] nil value at `volumes.persistentVolumeClaim.claimName` ignored in mutation attempt
clusterrole.rbac.authorization.k8s.io/kotsadm-role configured
clusterrolebinding.rbac.authorization.k8s.io/kotsadm-rolebinding configured
configmap/sentry created
secret/sentry-postgresql created
secret/sentry-redis created
secret/sentry created
service/sentry-postgresql created
service/sentry-redis-master created
service/sentry-redis-slave created
service/sentry created
deployment.apps/sentry-cron created
deployment.apps/sentry-postgresql created
deployment.apps/sentry-redis-slave created
deployment.apps/sentry-web created
deployment.apps/sentry-worker created
statefulset.apps/sentry-redis-master created
job.batch/sentry-db-init created
job.batch/sentry-user-create created
ingress.extensions/sentry-ingress created
persistentvolumeclaim/sentry-postgresql created
persistentvolumeclaim/sentry created
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.
the namespace from the provided object "default" does not match the namespace "sentry-pro". You must pass '--namespace=default' to perform this operation.

So let's update base/kustomization.yaml with our namespace to see if that fixes it:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: sentry-pro # added
resources:
- admin-console/api-deployment.yaml
- admin-console/api-role.yaml
- admin-console/api-rolebinding.yaml
- admin-console/api-service.yaml
- admin-console/api-serviceaccount.yaml
# - admin-console/kotsadm-bundle-0.yaml
- admin-console/kotsadm-deployment.yaml
- admin-console/kotsadm-role.yaml
 # etc etc etc

At the end of this, the apply works. I could probably have also done this namespace tweak in a downstream, so you could argue this falls on the end user, but I'd say it's better for things to work out of the box, which I think we could do by following our own advice and omitting namespace on all the admin-console resources.
Unfortunately this still leaves kotsadm-api in a crash loop:
$ kubectl logs -f kotsadm-api-7d48cdcb6d-r42xx -n sentry-pro
[2020-07-25T15:59:57.357] [INFO ] [TSED] - Call hook $beforeInit
[2020-07-25T15:59:57.360] [INFO ] [TSED] - Call hook $onInit
[2020-07-25T15:59:57.361] [INFO ] [TSED] - Build providers
[2020-07-25T15:59:57.384] [INFO ] [TSED] - Call hook $afterInit
[2020-07-25T15:59:57.387] [INFO ] [TSED] - Call hook $onMountingMiddlewares
INFO  [ 2020-07-25T15:59:57.407Z] (kotsadm-api/1 on kotsadm-api-7d48cdcb6d-r42xx): ensuring a local cluster exists
ERROR [ 2020-07-25T15:59:57.409Z] (kotsadm-api/1 on kotsadm-api-7d48cdcb6d-r42xx): you must set AUTO_CREATE_CLUSTER_TOKEN

Workaround step 3: adding a cluster token via downstream
Let's make a downstream that patches in a cluster token:
mkdir ~/sentry-enterprise/overlays/us-east-1
cat <<EOF > ~/sentry-enterprise/overlays/us-east-1/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: sentry-pro
bases:
- ../midstream
patches:
- ./patch-cluster-token.yaml
EOF
export RANDOM_STRING=not-actually-random-fixme
cat <<EOF > ~/sentry-enterprise/overlays/us-east-1/patch-cluster-token.yaml
apiVersion: v1
kind: Secret
metadata:
  name: kotsadm-cluster-token
stringData:
  kotsadm-cluster-token: "${RANDOM_STRING}"
EOF

Now we should have something like this to apply:
overlays
├── midstream
│   └── kustomization.yaml
└── us-east-1
    ├── kustomization.yaml
    └── patch-cluster-token.yaml

We can verify this works with a kustomize build
$ kubectl kustomize ~/sentry-enterprise/overlays/us-east-1
# ...
---
apiVersion: v1
kind: Secret
metadata:
  creationTimestamp: null
  labels:
    kots.io/kotsadm: "true"
    velero.io/exclude-from-backup: "true"
  name: kotsadm-cluster-token
  namespace: sentry-pro
stringData:
  kotsadm-cluster-token: not-actually-random-fixme
# ...

let's delete the previous secret so we can overwrite the value (no replace -k yet)
$ kubectl delete -n sentry-pro secret kotsadm-cluster-token
secret "kotsadm-cluster-token" deleted

$ kubectl apply -k ~/sentry-enterprise/overlays/us-east-1

Let's verify really quick that we have some data in there now
$ kubectl get secret -n sentry-pro kotsadm-cluster-token -o yaml

apiVersion: v1
data:
  kotsadm-cluster-token: bm90LWFjdHVhbGx5LXJhbmRvbS1maXhtZQ==
kind: Secret
metadata:
  annotations:

And it looks like now our kotsadm-api pod is running okay. Hopefully this will also fix the crashloop in kotsadm as it waits for the bucket to be created in minio:
[2020-07-25T16:18:40.666] [INFO ] [TSED] - Call hook $onReady
INFO  [ 2020-07-25T16:18:40.666Z] (kotsadm-api/1 on kotsadm-api-7d48cdcb6d-h5rt4): Ensuring bucket exists...
INFO  [ 2020-07-25T16:18:40.696Z] (kotsadm-api/1 on kotsadm-api-7d48cdcb6d-h5rt4): Server started...
[2020-07-25T16:18:40.696] [INFO ] [TSED] - Started in 359 ms

success!
It looks like now kotsadm is up and running, as well as our Sentry app pods:
$ kubectl get pod -n sentry-pro
NAME                                  READY   STATUS      RESTARTS   AGE
kotsadm-6f94cd7b77-8hf82              1/1     Running     8          24m
kotsadm-api-7d48cdcb6d-h5rt4          1/1     Running     5          7m24s
kotsadm-migrations-1595692343         0/1     Completed   0          24m
kotsadm-minio-0                       1/1     Running     0          24m
kotsadm-operator-59d477b795-xzmtz     1/1     Running     0          24m
kotsadm-postgres-0                    1/1     Running     0          24m
sentry-cron-576d54f477-v9g4s          1/1     Running     0          26m
sentry-postgresql-6d9fcf9f65-grh4w    1/1     Running     0          26m
# etc etc etc...

From here we can launch the admin console
kubectl kots admin-console -n sentry-pro
  • Press Ctrl+C to exit
  • Go to http://localhost:8800 to access the Admin Console
# (eventually)
  • Go to http://localhost:9000 to access the application

We still have to go through and upload the license etc, but once we've gone through the UI setup things seems to be humming along nicely and we can launch the Sentry app on localhost:9000

@marccampbell @markpundsack  getting some more questions on this one -- can I get an ack that y'all are aware of it?

Yeah, the admin console not respecting the namespace flag is a real pain and super confusing. Per Slack, it also seems that an environment variable of POD_NAMESPACE is needed to get some of the namespaces correct and that too is confusing.

https://github.com/replicatedhq/kots/blob/master/pkg/upstream/admin-console.go#L125 seems bad.

POD_NAMESPACE at the very least governs what the Namespace template function returns.

Yeah, unfortunately I think pull needs a lot of attention and refactoring.
It was originally written to support a weird workflow which we no longer actually need. It attempts to "sideload" the application so that the admin console can install.
Since this was written, we've introduced automated installs, and airgap installs to existing clusters.
IMO, kots pull should be a way to get the Admin Console manifests (and application metadata i.e. branding and kots.io/v1beta1, application needed for RBAC permissions), and generate the manifests for it. KOTS supports airgap installs and I think this will turn pull into a composable command instead of an overreaching command.

@marccampbell fwiw, I was directed to kots pull as the means of doing the initial deploy of a kots application via GitOps (Argo CD to be specific). For me, simply being able to convert to GitOps post-install isn’t quite enough.

@genebean understood.  There are definitely some bugs in this workflow we need to address before this is viable.

I think this is likely the same issue @dexhorthy mentioned, but when I try to apply the manifests generated by this I get the following error:

The ConfigMap "kotsadm-bundle-0" is invalid: metadata.annotations: Too long: must have at most 262144 characters

fwiw, I was directed to kots pull as the means of doing the initial deploy of a kots application via GitOps (Argo CD to be specific). For me, simply being able to convert to GitOps post-install isn’t quite enough.

I've worked around this limitation by kots install-ing an empty app, configuring GitOps, then switching to the real channel.

fwiw, I was directed to kots pull as the means of doing the initial deploy of a kots application via GitOps (Argo CD to be specific). For me, simply being able to convert to GitOps post-install isn’t quite enough.

I've worked around this limitation by kots install-ing an empty app, configuring GitOps, then switching to the real channel.

The large ConfigMap appears to be fixed. kotsadm-bundle-0.yaml currently clocks in at 123 KB by my count, which is well under the 1 MB Kubernetes limit for ConfigMaps. I'm able to kubectl apply it without issue.


Find all instances of Header: redirect and add exit() after

mistakes were made

Just checked branch dev, there the patch is still applied, however, in master it was replaced. Perhaps one more knowledgeable about git might have a clue how to fix this.

This issue has been resolved with the last merge from dev into master


Doesn't work

Hey,
I installed the plugin, but when I make the request multiple types the header cache-status field is always: "miss":
traefik.yml
experimental:
  plugins:
    cache:
      modulename: "github.com/traefik/plugin-simplecache"
      version: "v0.2.1"

cache.yml
http:
  middlewares:
    regenbogen-ice-api-cache:
      plugin:
        cache:
          path: /cache/regenbogenice
          maxExpiry: 60

router

and my requests:

I hope you can help me.

I can't get it to work either. Might be because Traefik Enterprise has a cache feature?

The reason could be that the original response doesn't have any cache-related info for the underlying module https://github.com/pquerna/cachecontrol. In case anyone is still interested I've forked this plugin and added the hack for this.


clSPARSE does not build with OpenCL 2.0 on Debian

Using the latest v<IP_ADDRESS> tag, clSPARSE fails to build on Debian testing / unstable with the following error message:
clsparse-<IP_ADDRESS>/src/library/include/clSPARSE-private.hpp:31:21: fatal error: CL/cl.hpp: No such file or directory

Despite explicitly passing -DBUILD_CLVERSION=2.0 to CMake.

We have soft plans to switch the library internals to boost.compute, which will remove references to cl.hpp (see #138).  There is no ETA on this though, we only have a proof-of-concept at this point.

Alright I will hold on then.

Any news on this?

The momentum to convert the internals to boost.compute has stopped; we don't have the resources at this time.  However, I am working on getting approval to check-in cl.hpp pushed through.

@ghisvail can you check if clsparse compiles on debian with #187 ? You may need to run cmake with -DUSE_SYSTEM_CL2HPP=ON.

@pavanky I'll try to find some time to look at this by the end of the week. Cheers for the heads-up.

I refreshed the packaging with https://github.com/clMathLibraries/clSPARSE/commit/9677ec36a3c670654caec19308bc9fe44cea46bd, but cannot get the build to succeed. This is the cmake command run by the package build process:
cmake ../src -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_LOCALSTATEDIR=/var -DBUILD_BENCHMARKS=OFF -DBUILD_CLVERSION=2.0 -DBUILD_TESTS=OFF -DSUFFIX_LIB=/x86_64-linux-gnu -DUSE_SYSTEM_CL2HPP=ON

and the build log output:
make[1]: Entering directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
/usr/bin/cmake -H"/<<PKGBUILDDIR>>/src" -B"/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu" --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/CMakeFiles" "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/CMakeFiles/progress.marks"
make -f CMakeFiles/Makefile2 all
make[2]: Entering directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
make -f library/CMakeFiles/clSPARSE.dir/build.make library/CMakeFiles/clSPARSE.dir/depend
make[3]: Entering directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
[  2%] Generating ../include/kernels/source-provider.cpp
cd "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/library" && /usr/bin/cmake -DCL_DIR="/<<PKGBUILDDIR>>/src/library/kernels" -DOUTPUT="/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/include/kernels/source-provider.cpp" -P /<<PKGBUILDDIR>>/src/cmake/cl2cpp_raw.cmake
-- running cl2cpp_raw
SpGEMM_EM_kernels
SpGEMM_ESC_0_1_kernels
SpGEMM_ESC_2heap_kernels
SpGEMM_ESC_bitonic_kernels
SpGEMM_computeNnzCt_kernels
SpGEMM_copyCt2C_kernels
atomic_reduce
blas1
control
conversion_utils
csrmm_adaptive
csrmm_general
csrmv_adaptive
csrmv_general
dot
elementwise_transform
matrix_utils
reduce
reduce_by_key
scan
sort_by_key_common
sort_by_key_int
sort_by_key_uint
cd "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu" && /usr/bin/cmake -E cmake_depends "Unix Makefiles" "/<<PKGBUILDDIR>>/src" "/<<PKGBUILDDIR>>/src/library" "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu" "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/library" "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/library/CMakeFiles/clSPARSE.dir/DependInfo.cmake" --color=
Scanning dependencies of target clSPARSE
make[3]: Leaving directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
make -f library/CMakeFiles/clSPARSE.dir/build.make library/CMakeFiles/clSPARSE.dir/build
make[3]: Entering directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
[  5%] Building CXX object library/CMakeFiles/clSPARSE.dir/clsparse-init.cpp.o
cd "/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/library" && /usr/bin/c++   -DBUILD_CLVERSION=200 -DCLSPARSE_INDEX_SIZEOF=4 -DclSPARSE_EXPORTS -I"/<<PKGBUILDDIR>>/src/library" -I"/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/library" -I"/<<PKGBUILDDIR>>/src/include" -I"/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/include" -I"/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/clsparseTimer"  -m64 -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wall -pedantic -Wdate-time -D_FORTIFY_SOURCE=2  -fPIC -fvisibility=hidden -fvisibility-inlines-hidden   -std=c++11 -pedantic -o CMakeFiles/clSPARSE.dir/clsparse-init.cpp.o -c "/<<PKGBUILDDIR>>/src/library/clsparse-init.cpp"
In file included from /<<PKGBUILDDIR>>/src/library/include/clSPARSE-private.hpp:26:0,
                 from /<<PKGBUILDDIR>>/src/library/clsparse-init.cpp:21:
/<<PKGBUILDDIR>>/src/include/clSPARSE.h:39:2: error: #error clSPARSE does not yet implement OpenCL 2.0 interfaces
 #error clSPARSE does not yet implement OpenCL 2.0 interfaces
  ^
In file included from /<<PKGBUILDDIR>>/src/library/include/clSPARSE-private.hpp:36:0,
                 from /<<PKGBUILDDIR>>/src/library/clsparse-init.cpp:21:
/<<PKGBUILDDIR>>/src/library/include/clSPARSE-2x.hpp: In member function 'void clsparseCsrMatrixPrivate::clear()':
/<<PKGBUILDDIR>>/src/library/include/clSPARSE-2x.hpp:181:46: error: 'rowBlocks' was not declared in this scope
         values = col_indices = row_pointer = rowBlocks = nullptr;
                                              ^
/<<PKGBUILDDIR>>/src/library/include/clSPARSE-2x.hpp:182:9: error: 'rowBlockSize' was not declared in this scope
         rowBlockSize = 0;
         ^
/<<PKGBUILDDIR>>/src/library/clsparse-init.cpp: At global scope:
/<<PKGBUILDDIR>>/src/library/clsparse-init.cpp:71:2: warning: extra ';' [-Wpedantic]
 };
  ^
/<<PKGBUILDDIR>>/src/library/clsparse-init.cpp:80:2: warning: extra ';' [-Wpedantic]
 };
  ^
/<<PKGBUILDDIR>>/src/library/clsparse-init.cpp:89:2: warning: extra ';' [-Wpedantic]
 };
  ^
/<<PKGBUILDDIR>>/src/library/clsparse-init.cpp:98:2: warning: extra ';' [-Wpedantic]
 };
  ^
library/CMakeFiles/clSPARSE.dir/build.make:69: recipe for target 'library/CMakeFiles/clSPARSE.dir/clsparse-init.cpp.o' failed
make[3]: *** [library/CMakeFiles/clSPARSE.dir/clsparse-init.cpp.o] Error 1
make[3]: Leaving directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
CMakeFiles/Makefile2:88: recipe for target 'library/CMakeFiles/clSPARSE.dir/all' failed
make[2]: *** [library/CMakeFiles/clSPARSE.dir/all] Error 2
make[2]: Leaving directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
Makefile:152: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'

@ghisvail can you remove explicitly passing BUILD_CLVERSION ?

@pavanky same outcome

@ghisvail I was able to compile the library using  the following patch after configuring with the following command on debian testing.
cmake ../src -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_LOCALSTATEDIR=/var -DBUILD_BENCHMARKS=OFF -DBUILD_TESTS=OFF -DSUFFIX_LIB=/x86_64-linux-gnu -DUSE_SYSTEM_CL2HPP=ON

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 0bdec76..bf1940a 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -18,6 +18,8 @@
 # PUBLIC keywords
 cmake_minimum_required( VERSION 2.8.12 )

+ADD_DEFINITIONS(-DCL_USE_DEPRECATED_OPENCL_1_2_APIS)
+
 if( CMAKE_GENERATOR MATCHES "NMake" )
        option( NMAKE_COMPILE_VERBOSE "Print VERBOSE compile/link msgs to the console" OFF )
        if( NMAKE_COMPILE_VERBOSE )

Thanks for looking at it. Then, it should be fine if we were able to compile, unless you want me to double-check ?

@ghisvail I sent in this PR: https://github.com/clMathLibraries/clSPARSE/pull/189 may be wait until it is resolved?
Can you also raise a debian bug to update the cl2.hpp header? Khronos moved the headers to a github repo now: https://github.com/KhronosGroup/OpenCL-CLHPP
The version available in debian testing is a bit buggy.

The version available in debian testing is a bit buggy.

Is the 2.0.9 release enough? Or do you need anything newer?

@ghisvail This commit seems to be the issue: https://github.com/KhronosGroup/OpenCL-CLHPP/commit/befe77b66fd3dd87ce9fef940e475cdc71d8bd9f
It was made after the release of 2.0.9. May be you can ask upstream to make a new release?

I have just sent the bug reports to the Debian maintainers of the OpenCL headers asking for an update. Meanwhile, I'll check with upstream whether they are happy to do a release. Worst case, this commit could be cherry-picked onto 2.0.9 I presume.

@pavanky I saw your PR was merged, which means clSPARSE can now probably be built in Debian.
Any plans to tag a new release out soon ?

I can confirm that the current tip of develop can be successfully built on Debian. Closing.

My bad. OpenCL 1.2 builds fine, but OpenCL 2.0 still fails.

You can get a build going with the following patch:
diff --git a/src/include/clSPARSE.h b/src/include/clSPARSE.h
index 0af7f5a..43e6bd5 100644
--- a/src/include/clSPARSE.h
+++ b/src/include/clSPARSE.h
@@ -36,7 +36,7 @@ extern "C" {
 #if( BUILD_CLVERSION < 200 )
 #include "clSPARSE-1x.h"
 #else
-#error clSPARSE does not yet implement OpenCL 2.0 interfaces
+//#error clSPARSE does not yet implement OpenCL 2.0 interfaces
 #include "clSPARSE-2x.h"
 #endif

diff --git a/src/library/include/clSPARSE-2x.hpp b/src/library/include/clSPARSE-2x.hpp
index 7a5f0aa..b3e79fe 100644
--- a/src/library/include/clSPARSE-2x.hpp
+++ b/src/library/include/clSPARSE-2x.hpp
@@ -178,8 +178,7 @@ public:
     void clear( )
     {
         num_rows = num_cols = num_nonzeros = 0;
-        values = col_indices = row_pointer = rowBlocks = nullptr;
-        rowBlockSize = 0;
+        values = col_indices = row_pointer = nullptr;
     }

     clsparseIdx_t nnz_per_row() const

and start a build with OpenCL 2.0, but it leads to more errors:
[ 22%] Building CXX object library/CMakeFiles/clSPARSE.dir/internal/data-types/csr-meta.cpp.o
In file included from /home/ghislain/debian/packages/clsparse/src/library/internal/clsparse-control.hpp:22:0,
                 from /home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:21:
/home/ghislain/debian/packages/clsparse/src/library/../clsparseTimer/clsparseTimer-device.hpp:75:45: warning: ignoring attributes on template argument ‘cl_uint {aka unsigned int}’ [-Wignored-attributes]
     typedef std::pair< std::string, cl_uint > idPair;
                                             ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp: In function ‘clsparseMetaSizeResult clsparseCsrMetaSize(clsparseCsrMatrix*, clsparseControl)’:
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:32:44: error: ‘matrix_meta’ does not name a type
         sizeResult.metaSize = static_cast< matrix_meta* >( pCsrMatx->meta )->rowBlockSize;
                                            ^~~~~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:32:55: error: expected ‘>’ before ‘*’ token
         sizeResult.metaSize = static_cast< matrix_meta* >( pCsrMatx->meta )->rowBlockSize;
                                                       ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:32:55: error: expected ‘(’ before ‘*’ token
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:32:57: error: expected primary-expression before ‘>’ token
         sizeResult.metaSize = static_cast< matrix_meta* >( pCsrMatx->meta )->rowBlockSize;
                                                         ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:32:76: error: ‘void*’ is not a pointer-to-object type
         sizeResult.metaSize = static_cast< matrix_meta* >( pCsrMatx->meta )->rowBlockSize;
                                                                            ^~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:32:90: error: expected ‘)’ before ‘;’ token
         sizeResult.metaSize = static_cast< matrix_meta* >( pCsrMatx->meta )->rowBlockSize;
                                                                                ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:37:30: warning: ignoring attributes on template argument ‘clsparseIdx_t {aka unsigned int}’ [-Wignored-attributes]
     clMemRAII< clsparseIdx_t > rCsrrow_pointer( control->queue( ), pCsrMatx->row_pointer );
                              ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp: In function ‘clsparseStatus clsparseCsrMetaCreate(clsparseCsrMatrix*, clsparseControl)’:
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:56:30: warning: ignoring attributes on template argument ‘clsparseIdx_t {aka unsigned int}’ [-Wignored-attributes]
     clMemRAII< clsparseIdx_t > rCsrrow_pointer( control->queue( ), pCsrMatx->row_pointer );
                              ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:59:5: error: ‘matrix_meta’ was not declared in this scope
     matrix_meta* meta_ptr = nullptr;
     ^~~~~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:59:18: error: ‘meta_ptr’ was not declared in this scope
     matrix_meta* meta_ptr = nullptr;
                  ^~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:62:33: error: ‘matrix_meta’ does not name a type
         meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                 ^~~~~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:62:44: error: expected ‘>’ before ‘*’ token
         meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                            ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:62:44: error: expected ‘(’ before ‘*’ token
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:62:46: error: expected primary-expression before ‘>’ token
         meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                              ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:62:65: error: expected ‘)’ before ‘;’ token
         meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                                                 ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:63:16: error: type ‘<type error>’ argument given to ‘delete’, expected pointer
         delete meta_ptr;
                ^~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:68:20: error: ‘matrix_meta’ does not name a type
     meta_ptr = new matrix_meta;
                    ^~~~~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:75:29: warning: ignoring attributes on template argument ‘cl_ulong {aka long unsigned int}’ [-Wignored-attributes]
         clMemRAII< cl_ulong > rRowBlocks( control->queue( ), meta_ptr->rowBlocks( ) );
                             ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp: In function ‘clsparseStatus clsparseCsrMetaDelete(clsparseCsrMatrix*)’:
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:5: error: ‘matrix_meta’ was not declared in this scope
     matrix_meta* meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
     ^~~~~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:18: error: ‘meta_ptr’ was not declared in this scope
     matrix_meta* meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                  ^~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:42: error: ‘matrix_meta’ does not name a type
     matrix_meta* meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                          ^~~~~~~~~~~
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:53: error: expected ‘>’ before ‘*’ token
     matrix_meta* meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                                     ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:53: error: expected ‘(’ before ‘*’ token
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:55: error: expected primary-expression before ‘>’ token
     matrix_meta* meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                                       ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:95:74: error: expected ‘)’ before ‘;’ token
     matrix_meta* meta_ptr = static_cast< matrix_meta* >( pCsrMatx->meta );
                                                                          ^
/home/ghislain/debian/packages/clsparse/src/library/internal/data-types/csr-meta.cpp:96:12: error: type ‘<type error>’ argument given to ‘delete’, expected pointer
     delete meta_ptr;
            ^~~~~~~~

You shouldn't have to change anything.. What is the error you are getting earlier ?

My bad. OpenCL 1.2 builds fine, but OpenCL 2.0 still fails.

If you mean cl2.hpp works fine with OpenCL 1.2 I don't see a problem with it. I don't think clSparse has been patched to work in OpenCL 2.0 only.
Why do you need to force it to use OpenCL 2.0 APIs ?

If you mean cl2.hpp works fine with OpenCL 1.2 I don't see a problem with it.

Indeed it works. Will it be sufficient for the next release of ArrayFire then?

I don't think clSparse has been patched to work in OpenCL 2.0 only.

That's what I figured from the results of the build above. Seems that the OpenCL 2.0 path of the codebase has not been adapted to recent changes in the API (such as https://github.com/clMathLibraries/clSPARSE/commit/e92f41b2269793e3c26b40429793e93bc3eabd4d).

Indeed it works. Will it be sufficient for the next release of ArrayFire then?

ArrayFire also targets 1.2. Considering that OpenCL libraries are backwards compatible, I think that is a fair.

That's what I figured from the results of the build above. Seems that the OpenCL 2.0 path of the codebase has not been adapted to recent changes in the API

I don't think clSparse ever had an OpenCL 2.0 path. @kknox correct me if I am wrong here.

ArrayFire also targets 1.2. Considering that OpenCL libraries are backwards compatible.

Then I'll stick with 1.2 for all clMath libraries to remain consistent with what AF supports.

I don't think clSparse ever had an OpenCL 2.0 path.

The official documentation suggests otherwise:
"OpenCL 2.0 support is not yet fully implemented; only the interfaces have been designed"


Doen't work with a folder structure

Hello there!
I really like this plugin. It helps me visualizing relations inbetween documents.
I work with folders to have an internal structure. The issue I'm facing now is that files that are not in the '/doc' (e.g. root folder) don't seem to be included in the build step of this plugin.
For example if I have files strucutred like this:

File1.md
Folder1

File2.md

File1 does get the backlinks but file 2 does not.
Am I using it wrong? Do I have to tweek some settings?
Greetings
Daniel

Hello Daniel,
It is supposed to get the backlinks in the structure you described. Here is an example of how I have setup my personal page: https://github.com/danodic-dev/danodicdev-digital-garden
Here is the configuration file: https://github.com/danodic-dev/danodicdev-digital-garden/blob/main/mkdocs.yml
Please let me know if your project follows the same structure.
I appreciate If you could please provide a sample of File1.md and File2.md, it may help on debugging the issue.
Thanks.

Hey
Sorry for the late reply. I didn't get around to respond.
I checked out your digital garden and it seemed to me that the backlinks as I intend to use them are also not working there.
My file structure looks as follows (Similar to yours):
/docs

index.md
xy.md
/a

a1.md
a2.md
a3.md
a4.md

/b

b1.md
b2.md
b3.md

/c

c1.md
c2.md

Now, if I link from a file in one folder to a file in another, the link doesn't get recognised. I tried to visualize the linkage by arrows:
flowchart LR
    subgraph docs
    index;
    xy
    subgraph a
    a1;a2;a3;a4;

    a1-->a2
    end

    subgraph b
    b1;b2;b3
    b1-->b2
    b1 --> a1
    end

    subgraph c
    c1;c2;
    c1-->c2
    c1 --> a4
    b3 --> c1
    end
end

For example; the link from b1 to a1 is not considered in the build step as it goes from one subfolder to another.
I'll have a look at your plugin any maybe come up with a solution.
saludos
Daniel

I found something the helped me. You normalize the links very nicely but some still contain backslashes ('\'). Afterward they cannot be found in the the dictionary you have.
This appears to be happening in my scenario with the subfolders.
As a quick fix I just appended replace('\', '/') to your normalize_link() method [line 74]
destination_link = self.__normalize_link(href, page.url).replace('\\', '/')

have a nice day

Thanks for the heads up.
It makes sense, as I have developed and testes in Linux where the slashes are all /.
I'll add this fix to it, thanks!


update webpack and react tutorial to webpack v2

Running this tutorial with Webpack v. >2.1 throws an error:
Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
 - configuration.module has an unknown property 'preLoaders'. These properties are valid:
   object { exprContextCritical?, exprContextRecursive?, exprContextRegExp?, exprContextRequest?, loaders?, noParse?, rules?, unknownContextCritical?, unknownContextRecursive?, unknownContextRegExp?, unknownContextRequest?, unsafeCache?, wrappedContextCritical?, wrappedContextRecursive?, wrappedContextRegExp? }
   Options affecting the normal modules (`NormalModuleFactory`).
 - configuration.resolve.extensions[0] should not be empty.

I have updated the tutorial and added the correct code for webpack v2.

Trying to write up how to use Webpack in our docs seems to have put us on a really fast treadmill that we're not actually equipped to keep up with. I'm going to close the old PRs relating to Webpack on the assumption that they're probably just as out-of-date as the content they're trying to update. Sorry to have let this sit around so long.


Bad signature#utils#SetupHighlightGroups behaviour when synIDattr returns -1

After updating to the latest version (with Fix for #116) I'm getting the following error when starting vim:
Error detected while processing function signature#utils#SetupHighlightGroups..CheckAndSetHL:
line    4:
E421: Color name or number not recognized: ctermbg=-1

I think this is the line where it goes wrong:
let l:color=synIDattr(synIDtrans(hlID(a:set)), a:what, a:prefix)

synIDattr returns -1 here, which I believe it shouldn't.
I found some Google Groups discussions mentioning the problem:

https://groups.google.com/forum/#!topic/vim_dev/rUyy1SjbkbU
https://groups.google.com/forum/#!searchin/vim_dev/synIDattr/vim_dev/S-x9R3NSi-c/ihv-vXAjFwAJ

These imply that it's not the intended behaviour, but apparently it's still not fixed, so maybe handling the -1 case is necessary here.
Does anyone else have this issue?

That's weird. I tested it with made up highlight group names as well as cleared ones and I got a blank string everytime. Anyways, hope this additional boundary check works.

Thanks, that fixed it for me


MAUI Workloads need to be updated.

Describe the bug
When running the pipeline, I get the following error:
##[warning]D:\a_work_tool\dotnet\sdk\6.0.411\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(35,5): Warning NETSDK1202: The workload 'android' is out of support and will not receive security updates in the future. Please refer to https://aka.ms/maui-support-policy for more information about the support policy.
Additional context
Only seen in tests on CI/CD.

Information from PG indicates that "given how fast mobile moves, supporting older versions of .NET and LTS does not exist for MAUI".  As a result, I'll not be fixing this until .NET 8 comes out (which is the next LTS release) and will just ignore the MAUI support issues.


Limit for API calls

Hi,
Within the documentation and the billing information, it says that the max number of API calls for the free tier is 60 API calls/minute. I've been testing out the API using finnhubClient.quote and I'm getting undefined and null back.
I haven't been hitting the API that much, so was wondering if the limit changed or maybe there's something on my end.
Thanks

Was getting the same issue as OP.  Initially it worked for maybe the first 10 calls and then I started getting null responses with the body saying I needed to sign up for a free account in order to gain access.  I was using the generated code from within FinnHub from my free account so I don't see what's wrong on our end here.

@19qiana I didn't dig into the specifics of the library after this issue, so I started using IEX Cloud and basically use fetch with promise chaining to get the appropriate the data I need. This simple solution has been working as a simple turnaround. I could try to make an API for it at some point

I ended up just writing wrapper code around the endpoint calls / made my own client.  It's not as pretty but it works


Replace verbatim text with NOT_YET_IMPLEMENTED

Summary
This PR replaces the verbatim_text builder with a not_yet_implemented builder that emits NOT_YET_IMPLEMENTED_<NodeKind> for not yet implemented nodes.
The motivation for this change is that partially formatting compound statements can result in incorrectly indented code, which is a syntax error:
def func_no_args():
  a; b; c
  if True: raise RuntimeError
  if False: ...
  for i in range(10):
    print(i)
    continue

Get's reformatted to
def func_no_args():
    a; b; c
    if True: raise RuntimeError
    if False: ...
    for i in range(10):
    print(i)
    continue

because our formatter does not yet support for statements and just inserts the text from the source.

Test Plan
None?

Current dependencies on/for this PR:

main

PR #4904   👈

This comment was auto-generated by Graphite.

@MichaReiser merged this pull request with Graphite.


Delta Backup

We would like to add a delta backup option for pgmoneta. Our eventual hope is to save time and space when doing backup by only receiving changed files and replacing the rest with symlinks. See more details at the discussion forum #146

This is a potential big task.
We know the LSN of the last backup, so we only need to get the changes since then. How, and how to apply is another matter.
Delta backups are important in order to support bigger installations, so this is a 0.8 task for sure.

Yes this is a big task. I'll take my time. I'll start with something easy and let's discuss in the forum about design choices and details.

As a starter, we'll work on support for the official delta/incr in postgres 17


how can i get http querystring

in health_check_server.py example
async def health_check(path, request_headers):
if path == "/health/":
return http.HTTPStatus.OK, [], b"OK\n"
if i pass a get param,how can i got it

i use regex to get it ,have a better way?

i use regex to get it ,have a better way?

I'd use https://docs.python.org/3/library/urllib.parse.html

I'd use https://docs.python.org/3/library/urllib.parse.html


Cucumber-Expression:  (Cucumber::Glue::ArityMismatchError)

Summary
Previously following step was working smooth, however with upgrade to cucumber 3.1.2, It throws error mentioned in current behavior section
Given(/^I create a foo (5.5|5.4|5.3|5.2) Subscription Allocation( with a( future dated)? subscription attached)?$/) do |version, sub_attached, future|
end
Expected Behavior
Ability to handle nested optional groups
Sample steps to be accepted by above definition are following-
I create a foo 5.4 Subscription Allocation
I create a  foo 5.5 Subscription with a subscription attached
I create a foo 5.3 Subscription with a future dated subscription attached
Current Behavior
Your block takes 3 arguments, but the Regexp matched 2 arguments. (Cucumber::Glue::ArityMismatchError)
Possible Solution
Providing the ability to handle nested Expressions
Steps to Reproduce (for bugs)

Implement the step definition given in summery
Add sample step to it
run cucumber

Context & Motivation
We have subscription step which we have written in given format.
Your Environment

Cucumber Version used: 3.1.2
Operating System and version: Linux
Ruby 2.5.1
Link to your project:

Can you paste a full stack trace please?

This feels like an incorrect usage of regex, but just my personal opinion.
Having nested blocks is fine, but you're then having them both as always capturing, which feels incorrect.
Looking at your original use case something like this would be better imo.
# Matches first bracket (forced), 2nd bracket non-captured ignored, 3rd/4th ignored
I foo 5.4 Sub
# Matches first bracket (forced), 2nd bracket non-captured observed, 3rd ignored, 4th captured
I foo 5.5 Sub with a subscription attached
# Matches first bracket (forced), 2nd bracket non-captured observed, 3rd captured, 4th captured
I foo 5.3 Sub with a future dated subscription attached

/^I foo (5.5|5.4|5.3) Sub (?:with a )?(future dated )?(subscription attached)?$/

ping @anandchavan


Fix ECONNRESET errors that happens when seeding the DB

When working on #82 , I encounter the following issue:
I now experience ECONNRESET errors randomly. Initially after submitting a random number of persons, now also affecting other objects like Organization.

ECONNRESET (Connection reset by peer): A connection was forcibly closed by a peer. This normally results from a loss of the connection on the remote socket due to a timeout or reboot. Commonly encountered via the http and net modules.

See https://nodejs.org/api/errors.html#errors_common_system_errors
[HPM] Rewriting path from "/api/organizations?organizationId=clinical-proteomic-tumor-analysis-consortium" to "/organizations?organizationId=clinical-proteomic-tumor-analysis-consortium"
[HPM] POST /api/organizations?organizationId=clinical-proteomic-tumor-analysis-consortium ~> http://localhost:8080/api/v1
[HPM] Rewriting path from "/api/organizations?organizationId=columbia-university" to "/organizations?organizationId=columbia-university"
[HPM] POST /api/organizations?organizationId=columbia-university ~> http://localhost:8080/api/v1
[HPM] Error occurred while trying to proxy request /organizations?organizationId=cincinnati-childrens-hospital-medical-center from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Error occurred while trying to proxy request /organizations?organizationId=celgene from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Error occurred while trying to proxy request /organizations?organizationId=clinical-proteomic-tumor-analysis-consortium from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Error occurred while trying to proxy request /organizations?organizationId=center-for-research-computing from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Error occurred while trying to proxy request /organizations?organizationId=columbia-university from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Rewriting path from "/api/organizations?organizationId=consejo-superior-de-investigaciones-cientificas" to "/organizations?organizationId=consejo-superior-de-investigaciones-cientificas"
[HPM] POST /api/organizations?organizationId=consejo-superior-de-investigaciones-cientificas ~> http://localhost:8080/api/v1
[HPM] Rewriting path from "/api/organizations?organizationId=corevitas" to "/organizations?organizationId=corevitas"
[HPM] POST /api/organizations?organizationId=corevitas ~> http://localhost:8080/api/v1
[HPM] Error occurred while trying to proxy request /organizations?organizationId=consejo-superior-de-investigaciones-cientificas from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Error occurred while trying to proxy request /organizations?organizationId=corevitas from localhost:4200 to http://localhost:8080/api/v1 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
[HPM] Rewriting path from "/api/organizations?organizationId=defense-advanced-research-projects-agency" to "/organizations?organizationId=defense-advanced-research-projects-agency"
[HPM] POST /api/organizations?organizationId=defense-advanced-research-projects-agency ~> http://localhost:8080/api/v1
[HPM] Rewriting path from "/api/organizations?organizationId=corrona" to "/organizations?organizationId=corrona"
[HPM] POST /api/organizations?organizationId=corrona ~> http://localhost:8080/api/v1
[HPM] Rewriting path from "/api/organizations?organizationId=dana-farber-cancer-institute" to "/organizations?organizationId=dana-farber-cancer-institute"

I'm almost sure that the issue happens because we sent to many request to the API service in parallel using RxJS forkJoin. ;-)
The two screenshots below show that the first few requests - tag creation - are fine with little to no time spent in "blocked" request state (red) and most in "waiting" (blue). The second image shows that requests spent most of their time being blocked (red) and ultimately the API service initiates the connection reset issue. I'm now looking at a way to throttle the number of concurrent submission (possibly using this approach).

I found an utility function that behaves like forkJoin and allows to specify the number of concurrent Observable to process. This solution is implemented in 0f500c1 to post 5 tags at a time. The first screenshot show that no request spend significant time in the "blocked" state. The second screenshot is obtained when settings the concurrency to 20. This leads some requests to spend time in the blocked state. Here the 7th request is the first one that spends time in the blocked state.

Using uWSGI (server side)
This time I decided to run the ROCC API service using Docker. This production-oriented approach leverages an uWSGI server placed in front of the Flask app to obtain better performance. The screenshot below shows that up to 20 tags can be pushed concurrently without any of them spending time in the blocked state. Note that slightly different results can be obtained each time the app is reloaded.

@thomasyu888 This ticket provides some insights into ECONNRESET errors that I encountered while working on the ROCC. This is the same issue that we encountered when seeding the NLP Sandbox Data Node and that we solved by introducing a delay between requests (I implemented both API services in the same way).
Here TypeScript-RxJS is a great solution for performing batch operations like seeding a databases.


Game has no frame rate control

While I was playing the game I found the game to be extremely fast and honestly unplayable because the badgers were moving so fast that I could never shoot them.
What is causing this
The game has no frame rate control, so it tries to run as fast as it can. So on my PC it is running at ~140 fps which is resulting in too many updates per second. So since the loop runs faster so do all the location updates of all the elements.
How to fix it
We can fix this by limiting the frame rate to something more reasonable like 30. This can be done using the pygame.time.Clock.tick() method.
I would like to take up this task.

I have submitted a PR for this issue please review it.

Thanks @saurabh-kumar-vit for reporting this with a detailed explanation, and deep-diving to find the root cause and the corresponding solution. 💯


feat: Add animation when deleting an item from a list

###Video
https://user-images.githubusercontent.com/46370525/160695021-84adb4f3-2b61-493e-9db0-180634ff7caa.mp4
Issue Resolved:
https://github.com/openfoodfacts/smooth-app/issues/1403

Animation for delete list item

Welcome @M123-dev


Added asnychronous test completed events

This PR adds events whenever a test is completed. Without it, you have to wait for all the results to be available before you get any feedback. When running long suites of tests, this makes it a lot more usable.
All TestResult objects are already processed as events by the VS Code extension, so I don't think any changes are necessary to the vscode extension. This does mean it'll see them twice, once from the asynchronous event and once from the RunTestResponse, but I don't think this has any bad effects.

@JoeRobich I've had this one hanging for a while, can you review or mention someone to review

@david-driscoll @mholo65 @filipw Anyone have any additional thoughts?

@JoeRobich Doesn't look like I can merge this myself.

@Samirat Thanks


Add acronyms

Some acronyms I thought were missing
CMV: Change My View
DAE: Does Anyone Else
ELI5: Explain Like I'm 5 (years old)
LSHMSFOAIDMT: Laughing So Hard My Sombrero Falls Off and I Drop My Taco
NSFL: Not Safe For Life (gory or gross)
YSK: You Should Know

@ravaan

Can you remove LSHMSFOAIDMT - This is too big to be an acronym.
Remove the content in brackets for NSFL.

Thanks!

Done.
IMO - LSHMSFOAIDMT is a hilarious one that should be there in the collection.


Fix invalid json of benchmark output

Now performance dashboard doesn't work because latest run produces invalid JSON output

https://line.github.io/decaton/commit-data/a3f14197b71c65a7aa8d83b78d5c9756dbf9ca89/tasks_100k_latency_10ms_concurrency_20-benchmark.json

This is because current version of async-profiler output Profiling started on stdout and there seems to be no way to disable this
To address this, I added --file option to the benchmark module to prevent async-profiler output and benchmark output are mixed

The invalid json broke performance dashboard so we will need to manually fix it and commit to gh-pages branch


Event page is breaking

Description
 The event page is not behaving as expected. Here, the Upcoming Events section is correctly rendered. But the Past Events section is blank. This shouldn't be the case.
Screenshots/testimonials

@ankiiitraj @tusharjain0022 @hariom1625 heads up!

On it!


Build libsdl2.a

I'm new to meson, so excuse me if this isn't a reasonable request, but would it be possible to build a static archive of libsdl2 as well?
I think it's as easy as adding this line to meson.build:
sdl2static = static_library('sdl2',
  all_sources,
  include_directories : core_inc,
  c_args : core_args,
  link_args : core_ldflags,
  dependencies : core_deps,
)

If that checks out, I'm happy to open a PR.

You can already do this by setting the default_library option to static.
Also, this is an old and deprecated repo. All issues should go against the mesonbuild/wrapdb repo instead.

Welcome! This is indeed a reasonable request. Fortunately, though, it's also possible without any modifications.
This is such an important and valued feature that meson has a special buildtype and global option for it: library(), which respects -Ddefault_library=static
(You may view all available options by running meson configure in the top of the source tree, though this is admittedly somewhat terse due to necessity.)
The default value of this setting is "shared" which is why it usually creates a shared library only (you can ask for "both" as well) and we generally encourage every library to empower users to select their preferred linking style rather than hardcoding "static_library" or "shared_library". Of course sometimes a library only works correctly as one of the two, which is why we support that.
So, this should already work out of the box for you.
If you have any more questions, feel free to ask. Remember that as per https://mesonbuild.com/index.html#community you can get in touch with us (the meson developers) as well as many experienced users for casual real-time conversation via IRC. This is the best way to get immediate advice or help, especially if you are not sure whether something is a bug or not.

Note that both static_library('sdl'), and library('sdl') with the default type set to static, will produce the same file (that is, 'libsdl.a') so you cannot use both at the same time or you will get an error due to conflicting targets.