Position Control Refactoring Part 2

Describe problem solved by this pull request
This is part 2 of the refactoring from the monster pr #12072 and should be easy to understand and review since the diff is manageable.
Describe your solution

Changing the acceleration and jerk in the vehicle_local_position_setpoint message to arrays because they most of the time are loaded into Vector3fs immediately anyways and it doesn't make sense to write x, y, z out every time. Same goes for position and velocity but the interface to flight review should be switched in one go and not message by message so I'll combine that with https://github.com/PX4/Firmware/issues/9331
The WeatherVane library needs just the last column of the attitude rotation matrix so it's not necessary to keep the whole attitude setpoint and pass the quaternion. I refactored it to take just that last column.
Since the attitude setpoint is not needed as a position control state it can be filled on the stack and published.

Test data / coverage
SITL tested except for the WeatherVane library but double-checked it gets the exact same data.
Additional context
Part 1 that goes before this: #13247

@MaEtUgR Can't wait for part 3 :nerd_face:

@bresch Thank you! 👍
Here is part 3: https://github.com/PX4/Firmware/pull/13262


Issue with server/headless environments

Running the code locally on a system with a GUI is no problem. But when trying to run on a server (running the same version of Ubuntu, but with no GUI), the following error is encountered:
$ python src/profile_to_plot.py -i data/prediction_multi.profile -g data/ground_truth_multi.profile -b test -s sample_0 genus
qt.qpa.xcb: could not connect to display
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, minimal, minimalegl, offscreen, vnc, webgl, xcb.

Aborted (core dumped)

Much googling lead nowhere.

This sounds like an issue with matplotlib defaulting to qt, you can try to add a matplotlibrc file like we do in sourmash: https://github.com/dib-lab/sourmash/blob/master/matplotlibrc
For conda it also helps to use matplotlib-base instead of matplotlib, I think it avoids bringing qt and all the Xorg libs (which also makes environments way smaller)

Thanks for the suggestion @luizirber! I’ll give that a try (though I think ete3 brings in qt as well, so might be two issues to address).

Turns out just adding os.environ['QT_QPA_PLATFORM']='offscreen' before importing matplotlib, seaborn, or ete3 did the trick. So no need to switch to matplotlib-base (though it may indeed make the env smaller, so may be worth doing)


enable setting column Levels properties

Changed order in which things are done when initializing. Now, all
the "holy trinity" columns and user-specified extra columns
are saved in the structure after calling saveEventsFile('init').
Next, the user can edit column properties before calling
saveEventsFile('open') which will write the JSON.
This way, Levels of e.g. trial_type can be set by the user. Moreover,
this gives the user the freedom to set the Levels as containers.Map
data type, which describes mapping between each level name and its
corresponding description. This is super useful because sometimes
levels can have cryptic names like '1' and '2', and so it's handy
to explain what each code means (e.g. 'left' and 'right' respectively).
Here is an example:
% allocate
logFile = [];

% We can define what extra columns we want in our tsv file beyond the
% BIDS holy trinity ('onset', 'duration', 'trial_type')
logFile.extraColumns = {'rhythm', 'trigger'};

% init logfile
logFile = saveEventsFile('init',cfg,logFile);

% change info about trial_type
logFile.columns.trial_type.Levels = containers.Map({'listen','tap'}, ...
                                                   {'listen without movement','tap finger with the pulse'});

% change units to miliseconds
logFile.columns.onset.Unit = 'ms';
logFile.columns.duration.Unit = 'ms';

% add info to the extra columns
logFile.extraColumns.rhythm.bids.Description = 'name of the rhythmic pattern';
logFile.extraColumns.rhythm.bids.Levels = {'unsyncopated','syncopated'};

logFile.extraColumns.trigger.bids.Description = 'EEG trigger value';
logFile.extraColumns.trigger.bids.Levels = containers.Map({'1','2'},{'low tone','high tone'});

% open tsv and write json for events
logFile = saveEventsFile('open',cfg,logFile);

I hope this makes sense. I'm not sure about the _stim file bids rules, so I tried not to mess things up too much.

@TomasLenc
Thanks for this. Will try to have a look at it this weekend to give you some feedback.
Adding some tick boxes below, but that's more for me to remember to do those on the PR later (though you are more than welcome to give it a go if you fancy):

[ ] update unit tests till the continuous integration of Travis passes
[ ] apply linter to make miss_hit happy
[ ] update jupyter notebook with example

@Remi-Gau thanks, let me know what you think of the proposed changes. If you decide they make sense, I'm happy to update the notebooks with examples.
And sorry for the failed checks...I need to learn more about how this works!

Finally taking the time to look into this.
I like the way this is going. Just a couple of point before I start an "actual" review.
tsv files have no header rows
I think there is one thing that this does is that this adds a header row to the TSV files of the stim files which is not OK by the BIDS standard.
This is because openFile calls printHeaderExtraColumns no matter if the file is a an event or a stim file.
A way to deal with this would be to create a isStim boolean field in the logFile structure and to set it to True or False depending on the initialization used.
Then printHeaderExtraColumns would only run is isStim is false.
stim files have no "holy trinity"
The other problem is that this stim files are NOT required to have onsets, durations and trial_type columns and this would make those fields "obligatory".
https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/06-physiological-and-other-continuous-recordings.html
So in this case too we could rely on logFile.isStim to decide what initializeFile should put in the logFile.columns.
Does that make sense?

[ ] update help section

by the way @TomasLenc if you want me to guide you on how to work with the unit tests and how you can use them to see what other changes will need to be added to the code, let me know. More than happy to explain it.

Thanks for the feedback @Remi-Gau, I've looked up the rules for _stim file and changed the way the .tsv is saved. As you suggested, there is now a "isStim" field that controls the behavior depending on whether the file was initialized as _events or _stim.
Not sure what your plan was regarding the actual saving within the _stim.tsv file. Assuming you may want to use the same idea as for the _events.tsv file, I patched parts of the code to make it possible (even when saving "event-by-event"). The way I did it is not very beautiful, so feel free to reject this :D

by the way @TomasLenc if you want me to guide you on how to work with the unit tests and how you can use them to see what other changes will need to be added to the code, let me know. More than happy to explain it.
Thanks, that would be super nice cause it hurts my heart to see everything failing after my PR :(( Did you want to link me to some tutorial, or perhaps you may have time to chat? Let me know, I'm aware you're busy!

by the way @TomasLenc if you want me to guide you on how to work with the unit tests and how you can use them to see what other changes will need to be added to the code, let me know. More than happy to explain it.

Thanks, that would be super nice cause it hurts my heart to see everything failing after my PR :(( Did you want to link me to some tutorial, or perhaps you may have time to chat? Let me know, I'm aware you're busy!

We can chat tomorrow if you have time.
In the mean time you can have a look at this.
https://the-turing-way.netlify.app/reproducible-research/testing.html
Actually they have expanded it so I should probably have a look too.

Thanks for the feedback @Remi-Gau, I've looked up the rules for _stim file and changed the way the .tsv is saved. As you suggested, there is now a "isStim" field that controls the behavior depending on whether the file was initialized as _events or _stim.
Not sure what your plan was regarding the actual saving within the _stim.tsv file. Assuming you may want to use the same idea as for the _events.tsv file, I patched parts of the code to make it possible (even when saving "event-by-event"). The way I did it is not very beautiful, so feel free to reject this :D

Will try to check it tonight.

@TomasLenc I have opened another PR #119
The plan for this one is just some doc to help people contribute to this repo.
I think we should have a section on writing / using code tests.
So if you have time to help with this. :wink: No pressure.

I promise will fix at the tests tomorrow.

@all-contributors please add @TomasLenc code, ideas

@Remi-Gau
I've put up a pull request to add @TomasLenc! 🎉

Feeling honoured

hey @TomasLenc I did a rebase of this PR on the dev branch to fix some merge conflict. So make sure you pull first before making any changes. :wink:

This saveEventsFile will require additional tests to define how it behaves when saving stim files but I am tempted to move that into another PR since the test suite we have requires quite a bit of refactoring anyway.

@all-contributors please add @TomasLenc test


Question \ FR - Ability to assign static Pod\Service\Job names to RuleBook Activations (K8s)

Please confirm the following

[X] I agree to follow this project's code of conduct.
[X] I have checked the current issues for duplicates.
[X] I understand that EDA-Controller is open source software provided for free and that I might not receive a timely response.

Feature type
Enhancement to Existing Feature
Feature Summary
Currently when a RuleBook Activation is created and enabled a default name is generated based on its internal ID and given to the created Pod, Service and Job Objects inside a K8s clusters.
When there is a use-case where a user wants to have a central, long-lived and unique RuleBook Activation, it would be nice if it could have some control over the generated Service Object name. This would make it much easier to create default Ingress\Mappings to expose the RuleBook endpoint.
Currently the RuleBook Activation needs to always be first manually created, and then can we see the generated name for the service so they can be later exposed outside of the cluster.
Steps to reproduce
Create a RuleBook Activation.
See the generated Pod, Service and Job names.
Current results
Pod, Service and Job Names follow the following naming convention:

Pod: activation-job-<ID>-XXX
Service: activation-job-<ID>-<PORT>
Job: activation-job-<ID>

Sugested feature result
Add the ability the statically define the generated resources names (at least Service), in the RuleBook Activation page, and have that name be used for the generated Service Object instead.
Additional information
No response

RFE: https://issues.redhat.com/browse/AAPRFE-821

Pod: activation-job-<ID>-XXX
Service: activation-job-<ID>-<PORT>
Job: activation-job-<ID>

In the recent implementation, these names are updated to use 2 different IDs:

Pod: activation-job-<Activation ID>-<Instance ID>-XXX
Service: activation-job-<Activation ID>-<Instance ID>-<PORT>
Job: activation-job-<Activation ID>-<Instance ID>

These names change simply by reactivating the Rulebook, which increases the difficulty of maintaining a stable connection when users want to create an Ingress as the frontend of the Service. I would like to see this RFE implemented.

Short of handling the entire flow and creating the Route/Ingress for the user for cases where there is a webhook triggering the rulebook, I think this is a good solution.
The user could specify a custom service name to be used when EDA creates the Service for the Activation.  It would be on the user to create a unique name for the Service here.
I think that we should also show the Service name on the details page for the resulting RulebookActivation.

Why this was not even thought of when first implemented is somewhat bemusing.
This feature in particular intorduced a bug, (#616) As the implementation was poorly done.  A simple understanding of the DNS would not have had this feature implemented the way it was.
It creates a service in this case, which exists to leverage the DNS system-Which in turn is a large part of how kubernetes communicates hence it's built in implementation of a complete system. DNS exists largely for the purpose of solving the "changing IP address problem". having an item "kubernetes Service" with a name that constantly changes defeats the purpose of having the service in the first place.
I touch upon a part of the issue in #616, with the "useless service name information"  whilst this useless information is only metadata, it's better suited for annotations. This still raises an issue with "A DNS name must be unique within the domain it forms part of." This problem however is only reduced to the namespace that EDA is part of, so just the service name. setting the service name to match the rulebook name would suffice with the EDA /rulebook metadata being added as annotations. Using this name as the service name you'd have to filter to [a-z|-] so it remains a valid dns hostname (the first octet of the DNS address). On the odd case that there are two rulebooks with the same name, this would be up to the administrator to correct; As the same name implies the same actions. Although now i think about it, possibly including the project name would allow two projects to have a rulebook with the same name.
As it currently stands now, this amoungst a few other "war stoppers" (As time permits you'll have my bug reports with solutions) makes EDA un-usable as now I need to have automations or a cron job setup to check and correct when a rulebook activation service name changes and as such the time taken to correct this will miss any inbound data, which is the sole purpose of having EDA in the firstplace.
I'm a +1 (and some) to have this prioritized for immediate resolution

We also wait for this to be resolved so we could go into production.

This issue is already solved, now activations can define a static service name for k8s.


How does Obsidian Remote work? Would it be possible to have an Obsidian server on my Mac without Docker?

Isn't Obsidian a closed source app? How are you able to take an Electron app, put it into a Docker container and turn it into a localhost server? Is there anything I could Google to learn more about the way it works, or anything to read about it? I did some googling on turning Electron apps into servers but didn't find good results.
Also, I've been trying to find a way to somehow run multiple instances of Obsidian on my Mac. What I'm going for is to have multiple vaults open simultaneously and I'd be able to access them in different app icons on my dock. If I opened two vaults in my one Obsidian app, it would show as 2 different windows which I don't want. Creating a 2nd copy of the app doesn't solve it either, it will just open in the same windows for some reason.
The thing about running it in a Docker container is that it would require running a Linux VM underneath and that would take up lots of resources.

The container just uses a linux image to load the obsidian app and then show it via a browser as a remote desktop app. Look though the docker file as it uses the linux server images as a base.

I'm not very familiar with Docker and haven't really used containers before. I looked through the repo and it looks like it installed some apt packages, and I found the autostart file that contains this:
/squashfs-root/obsidian --no-sandbox --disable-dev-shm-usage --disable-gpu --disable-software-rasterizer

Is this what launches Obsidian as a remote desktop app? I don't understand how that's possible. I'd still like to know where I could read about this or terms to look up. Would this be possible on a Mac, ideally without a Linux VM?
If it is like accessing an app from a remote desktop, how's the resolution and performance? Is it lower like comparing a RDP session to using the device in-person, if that makes sense?

 To run this you need to have an environment running docker. When the container starts it enables a website that if you connect to will show Obsidian in your browser. This is just a web version of RDP.

Oh I figured that out a while while waiting for a response, thanks anyway. Its simply an RDP session that shows in the browser. I thought this app was doing something special with Obsidian or something with Electron, something that it normally wouldn't do, to make it show up on as a website accessible in your browser.


feat: add e2e test

hi @helloanoop
since this awesome project doesn't have any e2e tests integrated. I propose to add one, using Playwright. 🙇🏻

Merged !
Thank you so much for this @depapp !
Glad to get to know about playwright.
And thank you so much for integrating github actions to run on pushes to main branch, as well as PRs to main branch.


cinnamon.cinnamon-session: 4.6.2 -> 4.8.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/cinnamon-session/versions.
meta.description for cinnamon.cinnamon-session is: "The Cinnamon session manager"
meta.homepage for cinnamon.cinnamon-session is: "https://github.com/linuxmint/cinnamon-session"
meta.changelog for cinnamon.cinnamon-session is: ""
Updates performed

Version update

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 4.8.0 with grep in /nix/store/jwb46knzfpspkdgr10v8amvybz1hyqdf-cinnamon-session-4.8.0

directory tree listing: https://gist.github.com/33daf8e08e7a8e9c4c7c8bdd408f9f0b

du listing: https://gist.github.com/6f246a9e09d5014cc3339e942c9f77b8

Rebuild report (if merged into master) (click to expand)

10 total rebuild path(s)

4 package rebuild(s)

4 x86_64-linux rebuild(s)
3 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
3 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
cinnamon.cinnamon-common
cinnamon.cinnamon-gsettings-overrides
cinnamon.cinnamon-screensaver
cinnamon.cinnamon-session

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/jwb46knzfpspkdgr10v8amvybz1hyqdf-cinnamon-session-4.8.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A cinnamon.cinnamon-session https://github.com/r-ryantm/nixpkgs/archive/54eed8ab5e4aea37c85e2b604a913ed35e0f064c.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/jwb46knzfpspkdgr10v8amvybz1hyqdf-cinnamon-session-4.8.0
ls -la /nix/store/jwb46knzfpspkdgr10v8amvybz1hyqdf-cinnamon-session-4.8.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review 1

4 packages built:

cinnamon.cinnamon-common
cinnamon.cinnamon-gsettings-overrides
cinnamon.cinnamon-screensaver
cinnamon.cinnamon-session

Maintainer pings
cc @mkg20001 for testing.

@SuperSandro2000 There's now 4.6 and 4.8 packages for cinnamon mixed up in unstable. I'm not sure that's a good idea. See https://github.com/NixOS/nixpkgs/issues/105622 for my original plans

Changes seem minimal so I thought it would be safe to merge them.
Waiting for the bot to create all PRs is not the best idea because it can take a while until it has created all packages.


PCNT units 4-7 not available on esp32 (IDFGH-4547)

Environment

Development Kit:      ESP32-DevKitC
Kit version (for WroverKit/PicoKit/DevKitC): v4
Module or chip used:  ESP32-WROOM-32D
IDF version (run git describe --tags to find it): v4.2
Build System:         idf.py
Compiler version (run xtensa-esp32-elf-gcc --version to find it): 8.4.0
Operating System:     macOS
(Windows only) environment type: [MSYS2 mingw32|ESP Command Prompt|Plain Command Prompt|PowerShell].
Using an IDE?: VSCode
Power Supply:         USB

Problem Description
PCNT units 4-7 are not available on esp32 after changes for esp32s2.  Number of units is defined in esp-idf/components/soc/soc/esp32/include/soc/pcnt_caps.h as SOC_PCNT_UNIT_NUM, which affects enum constants in esp-idf/components/soc/include/hal/pcnt_types.h
First problem: No other header file includes pcnt_caps.h, and it is not mentioned in documentation, so client-side code will never have the constants for all 8 units available.
Second problem: esp-idf/components/driver/pcnt.c includes pcnt_caps.h, but only after pcnt_types.h, so it has no effect, and the compiled code will not support units 4-7.  ("soc/pcnt_caps.h" must move to before "driver/pcnt.h".)
Expected Behavior
That PCNT units 4-7 work as usual on esp32
Actual Behavior
Compile or runtime errors for units 4-7
Steps to reproduce

Open examples/peripherals/pcnt/main/pcnt_example_main.c
Replace "PCNT_UNIT_0" with "PCNT_UNIT_4"
Attempt to compile
Compile fails

// If possible, attach a picture of your setup/wiring here.
Code to reproduce this issue
PCNT_UNIT_4

// If your code is longer than 30 lines, GIST is preferred.
Debug Logs
Debug log goes here, should contain the backtrace, as well as the reset source if it is a crash.
Please copy the plain text here for us to search the error log. Or attach the complete logs but leave the main part here if the log is *too* long.

Other items if possible

[ ] sdkconfig file (attach the sdkconfig file from your project folder)
[ ] elf file in the build folder (note this may contain all the code details and symbols of your project.)
[ ] coredump (This provides stacks of tasks.)

Thanks for reporting.

Thanks for reporting.

@TinkerBear This has been merged into release/v4.2 with 10f16c5d727c548a2e07 and should be available the next time we sync with Github. The issue will be automatically updated when this happens.

Thanks for reporting this!

@TinkerBear This has been merged into release/v4.2 with 10f16c5d727c548a2e07 and should be available the next time we sync with Github. The issue will be automatically updated when this happens.

Thanks for reporting this!

Thanks for reporting, the fix is available at https://github.com/espressif/esp-idf/commit/5e9783f4a6ffd96e016a6654a7bce2d12c5f6751, feel free to reopen. Thanks.


Talk model

Talk model. Added also lightning talk choice, just in case. If unwanted, please create an issue, it will be removed.

@mimmon there was already MR by @TomasHo who added talk model. However it didnt had the idea to have a lightning talk as a choice, which I like, so if you could update from upstream and amend the model. Thanks.

closing pull request, sent a new one


Feature/queue

As per #36 this PR adds in queue querying and controlling functionality.
You can now:

Clear the queue
Fetch all songs in the queue
Play any song in the queue
Listen for queue changes with change:queue

/cc @gmusic-utils/gmusic-js @twolfson

/cc @mmazzarolo

Sooo we have a fundamental unanswered question here... This is manipulating the UI in a manner that the user can see/disrupts any existing user interaction (i.e. opening the queue panel). I don't feel comfortable manipulating the UI as we stop becoming a low-level integration and start becoming a set of scripts.
I would prefer to find other ways (e.g. talking to Polymer application directly).

@twolfson The problem with interfacing with Polymer is that Google have made a whole bunch of custom polymer components for GPM and there API's are all private and hidden and obfuscated.  Every time Google update GPM the function names (that are obfuscated) change, so even if we could find the current obfuscated polymer method to call now it won't work the second Google update GPM.
IMO this is the least interfering UI experience you can make.  It at "worst" interferes with the UI for 40ms, the animation to show the queue takes 400ms.  So the worst thing that can happen is that you don't see 1/10th of the opening animation :+1:

Do we have proof that the data is obfuscated? In my experience, global objects and their properties aren't obfuscated since they must allow access to scripts outside of their scopes (e.g. other <script> tags)

I have witnessed the variables change over time as a while ago I was exploiting a method inside GPM but then the next update it function name changed.  As far as I can tell their are only two scripts that the entire GPM application so everything is scoped to a single file so they don't have to worry about scripts outside of scope.

That's still not evidence that the queue-related portion of the code will be dynamic; variables like USER_ID and USER_CONTEXT stay consistent =/

@twolfson It's impossible to prove that something won't change.  You can only assume that at some point it will, especially if it is unofficial and undocumented.
And FWIW the queue related parts of the code (that I can find by tracing) are nested inside function scopes and therefore aren't globally available.

Similarly the UI can change on us and historically selectors have broken in frustrating ways (e.g. upgrading users in groups https://github.com/twolfson/google-music-electron/issues/22#issuecomment-150988801). In the end, global objects and UI are more/less equally stable in the long term.
Can you link me to the queue related parts of the code?

Can you link me to the queue related parts of the code?

Not easily, it's all minified and I have the code at breakpoints in a chrome devtools extension that unminifies it for me.  (Better than the built in tools).  Give me a bit I'll see what I can do :+1:

@twolfson I can't find a way to link to a section of minified code from dev tools  so you'll have to find it yourself or believe me when I say we can't hook in to it :laughing:
Although we are technically using the UI (and changing it) if we do some math we can see that we are possibly interrupting the UI for <= 40ms.
This is somewhere between 2 - 3 frames of rendering (assuming a 60FPS render in chrome).  And this won't even affect usage of GPM because if they attempt to open the queue during those 2 - 3 frames it will still open, just 2 - 3 frames slower.

Maybe post the beautified source code to gist.github.com and link me to a line number? Or screenshot it? Or quote the code blocks that are good evidence?

@twolfson Hows this
https://gist.github.com/MarshallOfSound/3a5dc38575f6f0d0a2b8a9164df5af76
I've removed about 99000 lines for you there :laughing: but basically all the GPM logical is wrapped up inside a massive self calling function.  The queue logic (some of which I posted there) is also inside that scope

Cool, thanks. I'm still not entirely sold on this though =/ Google Music could optimize its references by assigning variables via pass-by reference (might be good if names are out of GZIP range).
I have a silly idea that might get us insight but I don't have time to explore it at the moment:
Run JSON.stringify(window) with something like json-stringify-safe and search for the current song id in the JSON blob. If it's not there, then we know it's not accessible via the global scope (at least not unless we invoke a function...)

@twolfson You can't JSON.stringify(window) because for some reason (I'm assuming Polymer magic) there is a circular dependency inside window.

That's why you would use json-stringify-safe ;)

@twolfson And that crashes Chrome :+1: :laughing:
Too much stuff for it too handle I think :D

Yea, prob needs to be done with some items omitted (e.g. _.omit) and max depth parameters =/

After strategic _.omit I can somewhat confirm that the queue isn't stored in the Window object.

Can you provide an example of the code you ran? (still haven't gotten around to this myself) =/

@twolfson Gimme a sec. I totally did it in dev tools so I didn't save it :laughing:

@twolfson This is interesting, one of the things I was omitting (it was full of circular dependencies appears to contain some interesting data).  I still think that path is unstable due to obfuscation.
But this path in GPM
window.APPCONTEXT.Go.q[0].hb.Eg

appears to be a list of all your playlists and their associated information :thought_balloon:

OK, so I just found a REALLY obscure path that appears to give the current queue.
window.APPCONTEXT.Go.h[0].Hi.queue.Yc.items

@twolfson You can fetch all the queue information through these methods
// First song in queue
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0]

// Proper song object
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].R()

// AlbumArt
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].Jc()

// Artist
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].Gb()

// Album
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].mc()

// AlbumArtist
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].cl()

// Lowercase Title
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].Qn()

// Lowercase artist
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].wj()

// Lowercase album
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].nm()

// Lowercase albumArtist
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].lm()

// Type / Genre of song
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].Re()

// Length of song in milliseconds
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].nf()

// UNKNOWN: Returns a number
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].ua()
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].Ima()

// Get play count
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].lv()

// Get rating (0=Neutral, 1=Down, 5=Up)
window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].cf()

// Get if playing
!!window.APPCONTEXT.Go.h[0].Hi.queue.qc()[0].Td[46]

The problem is.  There is no play method.  So we have read only access to the queue.  I'll keep exploring but the UI is probably the way to go to control GPM

Wow, nice work on the research :+1: imo the current playSong implementation is limited as the song must already be queued up =/

OK,so I am like 99.999% sure that the method that plays songs is not accessible by us.  It is a method on any object of type MO but the MO class is scoped (not accessible from the global object) and all instances of that class are not returned through any functions.
I stand by the UI implementation :+1:

Alright, if we are going to go with a UI implementaton, then I'm on the fence about keeping it in gmusic.js. One alternative would be to build a gmusic-ui.js which extends window.GMusic.
Part of my reasoning behind this would be as a JS library, I don't expect my UI to be altered unless I request it (e.g. toggle visualization should start playing visualizations). I'm thinking beyond the current queue implementation, for example when we wanted methods to play any song, album, or artist which required navigating pages and whatnot.

@twolfson Fair call, I had a feeling that's where this would end up.  I already have a framework for extending the GMusic prototype (I've been doing it for a while with mini-player and playlists :+1:
Are we in agreement then that we should make  gmusic-utils/gmusic-ui.js and put things like playlists and queue control in there?

Yep, let's pull the trigger on making gmusic-ui.js. If we ever discover ways to avoid UI altering methods, then we can always add them back in here =)
I have set up a team for gmusic-ui.js and gmusic-ui.js-admins as well as a repo. I will let you take over from here :ship:

@twolfson I just pushed up some magical playlist code if you wanna take a look at how magical people do things :+1: :laughing:
https://github.com/gmusic-utils/gmusic-ui.js

Why wasn't there a PR? That's the ideal venue for code review =(

@twolfson Urgh, I keep forgetting with these new repos to PR into an empty master :cry:
Let me set something up for you to look at

I usually do a placeholder set of files via a boilerplate and then do a PR onto that master. Otherwise, git can get mad screwy with default branches.
Example: https://github.com/underdogio/resolve-link/commit/668bc9846986781ac9ddd129bb5ad4731f7a87c6

@twolfson
Yeah, I had to setup some funky branches.  But it's up on GH now
https://github.com/gmusic-utils/gmusic-ui.js/pull/1/files


Legal Element Extraction

Hello, very great job! Thank you for your contributions!
Could you provide some more details on how to construct the legal element extraction dataset? If so, I would highly appreciate it!
By the way, I want to find out if you have encountered the decrease of general ability while fine-tuning?

Could you provide some more details on how to construct the legal element extraction dataset?

There are multiple open-source datasets that you can refer to, for instance, LEVEN and JointExtraction. Based on these, we use ChatGPT to provide some chain-of-thoughts.

By the way, I want to find out if you have encountered the decrease of general ability while fine-tuning?

Yes, but I cannot summarize a method to avoid this from happening. As some suggestions, you may increase the diversity of your prompts, and mix some other types of data in each fine-tuning. Still, this is largely trial-and-error.


[WIP][SQL] Refactor the implementation of SparkDateTimeUtils#toJavaDate

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

@LuciferYang If I may ask, why was this PR abandoned? I was hoping to be able to try out the 4.0 preview without --add-opens

Under normal circumstances, there is no need to add additional --add-opens when using 4.0 preview, because some --add-opens will be added by default during the job submission process.
If you are interested in this PR itself, you can try submitting it again :)

@LuciferYang I'm mostly looking for it to be solved for unit tests. I don't know why reflection would have been necessary in the first place here, so I don't see myself advocating for a PR to drop it -- but I definitely am hoping for it :)


[feature]Calico Network policy support for Windows Containers on AKS-HCI

Calico Network policy support for Windows Containers on AKS-HCI

This feature is completed. https://docs.microsoft.com/en-us/azure-stack/aks-hci/new-akshcicluster#-primarynetworkplugin.


Change Multipart Entity API's "rotate" methods to work as intended

This PR changes the rotate() methods of AbstractMultipartEntity to rotate the vectors via pitch and yaw only, which also fixes the bug where pitch and yaw rotations would not stack (as result of using quaternions, presumably).
It addionally adds the option to switch between degrees and radians without having to use *(float)Math.PI/180f every time.

Like so?

alright, done


Do not send empty min/max

Avoid sending 0 as default value for min/max
Delete value from TerminologyEntry

Accepted


Cannot shrinkwrap npm packages of a webpart solution generated with the yeoman generator 1.3.2

Category

[ ] Question
[ ] Typo
[X] Bug
[ ] Additional article idea

Expected or Desired Behavior
After scaffolding an SPFx webpart solution with the yeoman generator, one can generate an npm-shrinkwrap.json file in order to fix the node_modules tree.
Background information
The use of an npm-shrinkwrap.json file has been recommended for example by Chris O'Brien (see Avoiding dependency issues in SharePoint Framework (SPFx) development).
Our team has had a regression in the past where an SPFx solution didn't build anymore with a fresh install, among others on our build server. A new version of an indirectly referenced npm package had been released, which broke the solution. We thus decided to work with the npm-shrinkwrap.json file, but this approach has been unsuccessful until now with the yeoman generator 1.3.x.
We work with npm 4 (since npm 5 is not yet supported by the SPFx framework), but it's interesting to see that npm 5 generates a package-lock.json file by default. In npm 5, package-lock.json is usually the replacement of npm-shrinkwrap.json. The npm team recommends the use of a package lock file:

It is highly recommended you commit the generated package lock to source control: this will allow anyone else on your team, your deployments, your CI/continuous integration, and anyone else who runs npm install in your package source to get the exact same dependency tree that you were developing on. Additionally, the diffs from these changes are human-readable and will inform you of any changes npm has made to your node_modules, so you can notice if any transitive dependencies were updated, hoisted, etc.

Observed Behavior
When generating a react webpart with the yeoman generator 1.3.2 and npm 4, I cannot generate a shrinkwrap.json file. The command npm shrinkwrap fails with the following error:
npm ERR! Problems were encountered
npm ERR! Please correct and try again.
npm ERR! peer invalid<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>
npm ERR!      C:\Users\lmc\AppData\Roaming\npm-cache\_logs\2017-10-06T13_09_29_960Z-debug.log

Here is the content of the file C:\Users\lmc\AppData\Roaming\npm-cache\_logs\2017-10-06T13_09_29_960Z-debug.log:
0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\lmc\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'shrinkwrap' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>5 silly lifecycle<EMAIL_ADDRESS>no script for preshrinkwrap, continuing
6 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>7 silly lifecycle<EMAIL_ADDRESS>no script for shrinkwrap, continuing
8 verbose stack Error: Problems were encountered
8 verbose stack Please correct and try again.
8 verbose stack peer invalid<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>8 verbose stack     at shrinkwrap_ (C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\lib\shrinkwrap.js:128:15)
8 verbose stack     at Array.<anonymous> (C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\node_modules\slide\lib\bind-actor.js:15:8)
8 verbose stack     at LOOP (C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\node_modules\slide\lib\chain.js:15:14)
8 verbose stack     at C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\node_modules\slide\lib\chain.js:18:7
8 verbose stack     at done (C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\lib\utils\lifecycle.js:132:5)
8 verbose stack     at LOOP (C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\node_modules\slide\lib\chain.js:7:26)
8 verbose stack     at C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\node_modules\slide\lib\chain.js:18:7
8 verbose stack     at C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\lib\utils\lifecycle.js:329:20
8 verbose stack     at C:\Users\lmc\AppData\Roaming\npm\node_modules\npm\node_modules\graceful-fs\polyfills.js:284:29
8 verbose stack     at FSReqWrap.oncomplete (fs.js:123:15)
9 verbose cwd C:\Users\lmc\Documents\projects\2014-04 CFF\src\test\MySpfxSolution
10 verbose Windows_NT 10.0.14393
11 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\lmc\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "shrinkwrap"
12 verbose node v6.11.2
13 verbose npm  v4.6.1
14 error Problems were encountered
14 error Please correct and try again.
14 error peer invalid<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>15 verbose exit [ 1, true ]

Also, if I run npm list --depth=0, I get the following error:
npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>
Steps to Reproduce
Do NOT install @microsoft/generator-sharepoint globally (see Why you should consider installing the SharePoint Framework Yeoman generator locally), and run the following commands:
mkdir demo-bug
cd demo-bug
npm install<EMAIL_ADDRESS>yo @microsoft/sharepoint --solutionName "MySpfxSolution" --framework "react" --componentType "webpart" --componentName "MySpfxSolution" --componentDescription "MySpfxSolution web part" --environment "spo" skipFeatureDeployment false
rm -rf node_modules
cd MySpfxSolution
npm shrinkwrap

The second error mentioned is encountered when running
npm list --depth=0

Environment information
> npm list -g --depth=0

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

@lucmoco I think this is related to:
🐞 Generator v1.3 missing peer dependency? #890

@michel-weber you're right. It's likely the same problem. Two remarks:

#890 was opened with the generator 1.3.1, so the issue has not been fixed with 1.3.2
The npm list --depth=0 problem is not nice but it's not critical as the web part can still be generated. However the npm shrinkwrap problem is a show-stopper for us, which prevents us from using 1.3.x features.

This has been fixed and should be released in a patch soon.
You can work around this by adding "ajv" as a devDependency to your project.

I can confirm that the SharePoint Generator v. 1.3.4 fixes this problem.
Thanks for the good work !


Support for publishing libraries to mavenLocal

Fix for #542 and #545

Hey, thanks for catching this. It looks like we've missed the following section from the Pegasus doc: https://github.com/linkedin/rest.li/wiki/Gradle-build-integration#publishing-maven-artifacts . It looks similar to yours. What do you think about adding the "official" documentation?

I tried adding that last night and while it meant the data-template jar was uploaded to artifactory, the empty jar was still there.  The layout would then be:
gobblin-rest-api-VERSION.jar 				261 bytes
gobblin-rest-api-VERSION-sources.jar 			261 bytes
gobblin-rest-api-VERSION-javadoc.jar 			261 bytes
gobblin-rest-api-VERSION-data-template.jar 	41 kilobytes
I’m not confident that this would actually solve the problem.  In addition, it seems misleading that the jar, sources.jar and javadoc.jar are empty.

On Dec 18, 2015, at 2:27 PM, Chavdar Botev<EMAIL_ADDRESS>wrote:
Hey, thanks for catching this. It looks like we've missed the following section from the Pegasus doc: https://github.com/linkedin/rest.li/wiki/Gradle-build-integration#publishing-maven-artifacts https://github.com/linkedin/rest.li/wiki/Gradle-build-integration#publishing-maven-artifacts . It looks similar to yours. What do you think about adding the "official" documentation?
—
Reply to this email directly or view it on GitHub https://github.com/linkedin/gobblin/pull/544#issuecomment-165911372.

The build failure should be fixed by #548

@kadaan can you have a look at https://github.com/linkedin/gobblin/pull/550 ? This generates two more artifacts gobblin-rest-api-data-template and gobblin-rest-api-rest-client . gobblin-rest-api.jar is empty because there is no code in src/main/java . The concern that I have about your PR is that if someone adds code under src/main/java , it will be dropped since gobblin-rest-api.jar gets overwritten.


FizzBuzz in ASCII Art

A FizzBuzz program, but the output is displayed in ASCII art.
I would recommend not to keep a huge font type , otherwise the ASCII art wont look as it is supposed to

Congrats! This is by far the longest solution we got so far :')


Default basic auth password is reset during upgrade

We have a config on the master where the admin can set the basic auth password:
https://github.com/kubernetes/kubernetes/blob/master/cluster/juju/layers/kubernetes-master/reactive/kubernetes_master.py#L264
During a charm upgrade the password_changed() method will be called with password set to "". We do not want to reset the password if it's already set. Therefore we have a state 'client.password.initialised' that acts as a guard flag. The 'client.password.initialised' is set on the leader but it is not passed to other non-leader master nodes. So if you remove the leader you also unset the state.
What does this mean: if you do not change the random password set during deployment, you remove the master leader and then upgrade the charm you should get a new random admin password. Haven't tested this yet, just reviewed the code.

False alarm. 'client.password.initialised' is guarding the case where we are upgrading from a charm that did not have the client-password config at all and the password_changed() method is called more than once.


Out-of-date

The screenshots are out of date and doesn't include Account Key for authentication. We now also have the extra options around Data Views - either File System View or CDM Folder View

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: de89bef7-dbd7-c901-d329-f9838f8dbaf9
Version Independent ID: 583cc4ae-3fdb-0d6c-6314-8aaf8e51e922
Content: Analyze data in Azure Data Lake Storage Gen2 by using Power BI
Content Source: powerquery-docs/Connectors/DataLakeStorage.md
Service: powerquery
GitHub Login: @bensack
Microsoft Alias: bensack

@matt40k This issue has been fixed.


test: test fail-argument

The cause of this issue is trivial: $INPUT_FAIL gets changed to true because of failIfEmpty: true, which is the default. The line INPUT_FAIL=true was introduced in #86.
https://github.com/lycheeverse/lychee-action/blob/ae4699150ab670dcfb64cc74e8680e776d9caae2/entrypoint.sh#L34-L39
https://github.com/lycheeverse/lychee-action/blob/ae4699150ab670dcfb64cc74e8680e776d9caae2/action.yml#L17-L19
To verify it, I set failIfEmpty: false to the step in test.yml, and now it does pass.
  - name: test disable fail - it's okay if we fail
    uses: ./
    with:
      args: --no-progress foo.bar
      fail: false
+     failIfEmpty: false

Thanks https://github.com/nektos/act for making it a lot easier to debug.
Possible fix
Document this behaviour, or edit entrypoint.sh:
	# Overwrite the exit code in case no links were found
	# and `failIfEmpty` is set to `true` (and it is by default)
	if [ "${INPUT_FAILIFEMPTY}" = "true" ]; then
-	    # Explicitly set INPUT_FAIL to true to ensure the script fails
-	    # if no links are found
-	    INPUT_FAIL=true
	    # This is a somewhat crude way to check the Markdown output of lychee
	    if grep -E 'Total\s+\|\s+0' "${LYCHEE_TMP}"; then
	        echo "No links were found. This usually indicates a configuration error." >> "${LYCHEE_TMP}"
	        echo "If this was expected, set 'failIfEmpty: false' in the args." >> "${LYCHEE_TMP}"
	        exit_code=1
+	        # Explicitly set INPUT_FAIL to true to ensure the script fails
+	        # if no links are found
+	        INPUT_FAIL=true
	    fi
	fi

I can make a PR if wanted.

Thanks for the investigation. I'm all for changing the code here. The way I see it,

If fail: false and failIfEmpty is not set, don't fail if the status code is not 0 but fail if links were found.
If fail is not set, and failIfEmpty: false, fail if the status code is not 0 but don't fail if there are no links.
If none is set, use the defaults: fail if the status code is not 0 or no links were found.

Please correct me if I'm wrong.
I'd appreciate a pull request.

Maybe this PR can be merged first?
If #262 needs more tests, but this PR is not merged, then there will be a merge conflict later.

Maybe this PR can be merged first?

Do you mean this PR? (251)
Because it currently fails.
I checked out your PR (https://github.com/lycheeverse/lychee-action/pull/262) and that passes and seems to fix the issue, no? I think we can close this one (251) and merge yours. Or am I missing something?

Ah, I think I understand now. Can you merge the changes of this PR into your branch? Then we have all changes (including a passing test) in one pull request. I prefer that, because it's a single atomic change and we can make sure it works. I'd rather not merge a change if the pipeline is red. 😉

@sebastiaanspeck, this PR got merged as part of #262 by @YDX-2147483647.
Good work by both of you. 🎉


Test-DbaReplicationLatency

Is this a feature OR bug:
Enhancement
https://trello.com/c/4hjmza3B
Additional comments
Needs to be fleshed out or closed.

@sql-williamd you up for this still?

@potatoqualitee sorry - totally forgot this. The trello bard is gone..... any idea what the contents of that were?

@sql-williamd trello card had no specific details. Looks like it was one @js0505 had created though so maybe he has further ideas/thoughts on it.

perhaps https://www.mssqltips.com/sqlservertip/2590/determine-data-latency-between-publisher-and-subscriber-in-sql-server-transactional-replication/ or https://www.mssqltips.com/sqlservertip/3598/troubleshooting-transactional-replication-latency-issues-in-sql-server/ too


Add release notes and latest version to webpage

@NiLuJe, Can you take a quick look at it: https://pgaskin.net/kobopatch-patches?

I'd make the headline a link to the landing page (to complement the "Back" link), but besides this small nit, LGTM ;).

@NiLuJe, it seems that GitHub now supports linking to a file in the latest release, like github.com/geek1011/kobopatch-patches/releases/latest/download/kobopatch_4.18.13737.zip .


capture currentURL instead of using index to access to avoid race condition

New Pull Request Checklist

[x] I have read and understood the CONTRIBUTING guide

[x] I have read the Documentation

[x] I have searched for a similar pull request in the project and found none

[x] I have updated this branch with the latest master to avoid conflicts (via merge from master or rebase)

[x] I have added the required tests to prove the fix/feature I am adding

[x] I have updated the documentation (if necessary)

[x] I have run the tests and they pass

[x] I have run the lint and it passes (pod lib lint)

This merge request fixes / reffers to the following issues: #2110
Pull Request Description
When user call cancelPrefetching from different queue, maybe that completion block in loadImageWithURL: are still processing. Then the index previously used will pointer to the our of range index. So we should better capture the NSURL object and avoid using index again.

Codecov Report

Merging #2111 into master will increase coverage by 0.07%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##           master    #2111      +/-   ##
==========================================
+ Coverage   78.34%   78.41%   +0.07%
==========================================
  Files          35       35
  Lines        3389     3387       -2
==========================================
+ Hits         2655     2656       +1
+ Misses        734      731       -3

Impacted Files
Coverage Δ

SDWebImage/SDWebImagePrefetcher.m
79.48% <83.33%> (+3.23%)
:arrow_up:

SDWebImage/SDImageCache.m
64.73% <0%> (-0.53%)
:arrow_down:

SDWebImage/SDWebImageManager.m
78.35% <0%> (+1.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5872bd6...da7f3e4. Read the comment docs.

Merged...Tested it with a background queue by 100 requestes and seems OK. Attention that currently thread-safe is rely on that SDWebImageManager completion block called on main queue(serial of cause) to modify some requestedCount value. All outside acesss are forced to use @synchronized. Maybe we should refactor this to provide a more solid solution. :)


[AutoMM] GroundingDino used in AutoMM doesn't support CUDA 12 and Torch 2.1

Currently we are installing groundingdino from here. However, it got issue with Torch 2.1 and CUDA 12. See log

I believe this can be resolved by installing the same version of CUDA as used in PyTorch. Your error log tells you that PyTorch is using 11.7 so you need to install that. I ran into the same error. Installing a matching version of CUDA via apt, adding /usr/local/cuda-11.x to the path and setting CUDA_HOME correctly via .bashrc, and then adding the correct python headers solved this for me.

@rxjx Thanks for the suggestion! This custom installation is automatic and may cause obstacles for some users with less such knowledge. We will deprecate the current groundingdino implementation and add it back through the MMDetection support later.

Fixed by # 3974


Telekinesis force implant

About the PR
Telekinesis implant that gives ability to use personal Tether gun.
Why / Balance
Tether gun as implant is fun.
Technical details
Shit code.
Media

https://github.com/user-attachments/assets/cef92a82-3304-4f84-80dc-94273320d91d
Requirements

[x] I have read and I am following the Pull Request Guidelines. I understand that not doing so may get my pr closed at maintainer’s discretion
[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl:

add: Telekinesis ability implant now exists in game. Only by admins,

Honk mother bless this code, should be moved between files and renamed to appropriate names though

rename the "Telekinesis implanter" to "telekinesis implanter" i am BEGGING

Does it share code with tether gun?

Does it share code with tether gun?

no, it's basically dublicates Tether gun's code, no sharing

Does it share code with tether gun?

no, it's basically dublicates Tether gun's code, no sharing

Guh

They won't merge it if there is a lot of code duplication.
You gotta rework the pre-existing telekinesis system to work for the gun and for the implant.
You probably don't have to write any new code, just make the virtual blue hand item you are spawning with the implant button have the right components to work as a tether gun using the pre-existing code.

They won't merge it if there is a lot of code duplication.
You gotta rework the pre-existing telekinesis system to work for the gun and for the implant.

Yeah, this

Hello! Thank you for your contribution. Due to a lack of updates and the need for a full code rewrite this PR will be closed. If you return to implementing this functionality, feel welcome to make a new PR. Thank you!


Bug with CLOBS

Hi Rogier,
bug_test.zip
as you can see using example attached, there is a problem in
string.c
function unsigned int OCI_StringGetFromType
case OCI_CDT_LOB:

If you comment
putenv( "NLS_LANG=.AL32UTF8" );
all goes ok
I think about the declaration of
otext lob_buf[OCI_SIZE_BUFFER + 1 ];
this can be
otext lob_buf[OCI_SIZE_BUFFER + 4 ];
P.S:
See folder fix_vs2010
Using VS2010 with last release, the declarations of variables in functions:
boolean OCI_API OCI_Initialize
OCI_Mutex * OCI_MutexCreateInternal

Must be moved on top of the function, else there is a C Compiler errors.
Thanks for your patience.

Issue Fixed :)
Regards,
Vincent


Enable gzip compression

Description
Enables gzip compression as described here
Checklist

[ ] Make sure its compressed

Issue(s)
Addresses https://github.com/seattle-uat/civiform/issues/1940

I'm getting an error, need to work that out. Also should probably set the highest compression level

(sorry, that was a mistake)

I need to try it out in lighthouse still or something

Realized I had gzip disabled in dev. After enabling it in dev I still get the same thing in the report
Here are full reports
disabled_in_dev.pdf
enabled_in_dev.pdf

With default compression (level 6)
With gzip, tailwinds is 173kb bytes
Without, it is 1.8 Mb
So it's around a 10x reduction in size.  Pretty good!
On compression level 9, it's 171kb - probably not worth the extra processing time (though it is minimal)

I finally have a report on how gzip effected transfer size and all. Please see the updated description


Search (or add) not working.

Hello,
I'm following the tutorial from this book https://www.manning.com/books/cross-platform-desktop-applications.
They are using the old API, so I've tried using the new one but it does not work for me.
I cannot confirm if search and add exist from the perspective of API.

I'm not sure how to debug it. Any thoughts?
const lunr = require('lunr');
let index;

function resetIndex() {
  index = lunr(function () {
    this.field('file');
    this.field('type');
    this.ref('path');
  });
}

function addToIndex(file) {
  // The file is an object
  index = lunr(function () {
    this.field('file');
    this.field('type');
    this.ref('path');

    this.add(file);
  });
}

function find(query, cb) {
  if (!index) {
      resetIndex();
  }

  const results = index.search(query);
  // The results returns an empty array [] at this point
  cb(results);
}

It looks like you are using Lunr 2.x which no longer supports adding documents to an index after it has been built (more info in the upgrading guide).
That said it looks like you are working around this by rebuilding the index every time you want to add a file to it. At a glance it looks like you call resetIndex before doing the search, which doesn't actually add any files to the index. I think the problem then is in how you are creating the index before searching, you would need to ensure that all the files you want to search are being added to the index.
You can help debug the issue by checking if the index is empty in you find function:
index.tokenSet.toArray()

The above will return all the words that are in the index, if this is empty the either you didn't add anything to the index when building it, or the field object you did add to the index had no words.
Hopefully this helps you get to the bottom of the problem, let me know if you manage to come good on this.
P.S. Paul was a colleague of mine when I worked in London, great guy. Hope you are enjoying his book!

Thanks for your response.
I'll close this issue since I'm no longer working on this.


Initialize new private data asset transfer CC

Reworked private data sample as part of asset transfer series.
Signed-off-by: NIKHIL E GUPTA<EMAIL_ADDRESS>

This PR has two slightly different copies of privateAssetTransfer.go!

Nikhil,
We are starting to write application for private-data. On comparing with asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go
this is missing this chaincode func
func (t *SimpleChaincode) TransferAssetBasedOnColor(ctx contractapi.TransactionContextInterface, color, newOwner string) error
My thought is to either

Take asset_transfer_ledger_chaincode.go , and add private data, to demonstrate it in addition to couchdb queries
Or take asset_transfer_basic , and add private data. That will keep it simpler. So this will be without any ledger queries chaincode.

Thoughts on what will chime better for the Samples overall?

Nikhil,
We are starting to write application for private-data. On comparing with asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go
this is missing this chaincode func
func (t *SimpleChaincode) TransferAssetBasedOnColor(ctx contractapi.TransactionContextInterface, color, newOwner string) error
My thought is to either
* Take asset_transfer_ledger_chaincode.go , and add private data, to demonstrate it in addition to couchdb queries

* Or take asset_transfer_basic , and add private data. That will keep it simpler. So this will be without any ledger queries chaincode.

Thoughts on what will chime better for the Samples overall?

TransferAssetBasedOnColor is not possible with private data. You cannot do a range query that is used to update private collections, since private data may be missing on some peers we can't ensure that each peer returns the same results. This is mentioned in the private data documentation.

 This already stands at ~150 comments
If these chaincode is running fine, I was hoping this PR be merged, and Apps dev can start using this chaincode, unless you are close to the new revision

Are you doing Idiomatic Go in this PR itself? This already stands at ~150 comments
If these chaincode is running fine, I was hoping this PR be merged, and Apps dev can start using this chaincode, unless you are close to the new revision

Ill do what I can until its merged. If it is merged, I will do some clean up in a subsequent PR.

Thanks Nik, I'm going to merge and then you and Sijo can address remaining comments in subsequent PRs.


Shadow color

A feature to specify a custom shadow color would be nice.

Second this. Would love to have this


"script" command on linux contains gibberish characters

Asked on unix.stackexchange.com.

The following maybe worth trying next time (ref):
export LESS="-R"


Can't build on 14.04

When I try to build for fmu-v4 I get the error:
/home/carl/Wingtra/Standard_PX4/Firmware/src/modules/systemlib/px4_macros.h:91:54: error: expected ',' before ')' token
 #  define CCASSERT(predicate) static_assert(predicate)
                                                      ^
/home/carl/Wingtra/Standard_PX4/Firmware/src/systemcmds/hardfault_log/hardfault_log.c:137:1: note: in expansion of macro 'CCASSERT'
 CCASSERT(TIME_FMT_LEN == HEADER_TIME_FMT_LEN);
 ^
compilation terminated due to -Wfatal-errors.
make[3]: *** [src/systemcmds/hardfault_log/CMakeFiles/systemcmds__hardfault_log.dir/hardfault_log.c.obj] Error 1
make[2]: *** [src/systemcmds/hardfault_log/CMakeFiles/systemcmds__hardfault_log.dir/all] Error 2

I just re-cloned the repo and now I get the error:
rsync: stat "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/apps/include/graphics/.tiff.h.nGf1yb" failed: No such file or directory (2)
rsync: rename "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/apps/include/graphics/.tiff.h.nGf1yb" -> "apps/include/graphics/tiff.h": No such file or directory (2)
rsync: stat "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/apps/nshlib/.Kconfig.giwYAn" failed: No such file or directory (2)
rsync: rename "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/apps/nshlib/.Kconfig.giwYAn" -> "apps/nshlib/Kconfig": No such file or directory (2)
rsync: stat "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/configs/olimex-strp711/src/.str71_spi.c.bsXFpE" failed: No such file or directory (2)
rsync: rename "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/configs/olimex-strp711/src/.str71_spi.c.bsXFpE" -> "nuttx/configs/olimex-strp711/src/str71_spi.c": No such file or directory (2)
rsync: stat "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/configs/stm32f429i-disco/nsh/.setenv.sh.MzDQbJ" failed: No such file or directory (2)
rsync: rename "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/configs/stm32f429i-disco/nsh/.setenv.sh.MzDQbJ" -> "nuttx/configs/stm32f429i-disco/nsh/setenv.sh": No such file or directory (2)
rsync: stat "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/drivers/sensors/.zerocross.c.vR1c9R" failed: No such file or directory (2)
rsync: rename "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/drivers/sensors/.zerocross.c.vR1c9R" -> "nuttx/drivers/sensors/zerocross.c": No such file or directory (2)
rsync: failed to set permissions on "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/net/icmpv6/.icmpv6_input.c.8iXJ0W": No such file or directory (2)
rsync: rename "/home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/build_px4fmu-v4_default/px4fmu-v4/NuttX/nuttx/net/icmpv6/.icmpv6_input.c.8iXJ0W" -> "nuttx/net/icmpv6/icmpv6_input.c": No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.0]
make[3]: *** [nuttx_copy_px4fmu-v4.stamp] Error 23
make[2]: *** [CMakeFiles/nuttx_patch_Fixed-Shadow-wanings.patch-px4fmu-v4.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
[  3%] Applying /home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/nuttx-patches/ctype_cctype_.patch
[  3%] Built target nuttx_patch_ctype_cctype_.patch-px4fmu-v4
[  3%] Applying /home/carl/Wingtra/Standard_PX4_newnuttx/Firmware/nuttx-patches/serial_dma_hotfix.patch
[  3%] Built target nuttx_patch_serial_dma_hotfix.patch-px4fmu-v4
[  3%] Built target xml_gen
make[1]: *** [all] Error 2
make: *** [px4fmu-v4_default] Error 2

this looks like a duplicate of https://github.com/PX4/Firmware/issues/6128

Yea I wasn't sure, please feel free to close the issue if it is

if a ninja build works for you, then it's the same issue

@dagar Yet another issue.

Something weird with the nuttx copy (rsync). Trying to reproduce.

@CarlOlsson should be resolved on master. Can you verify?

Yes it works now, thanks!


join on multiple columns (?)

Is it possible to write something like the following with Esqueleto? Sorry if I've missed something.
SELECT *
FROM table1
INNER JOIN table2
ON table1.column1_name = table2.column1_name
AND table1.column2_name = table2.column2_name;

Yeah, just do something like:
select $
from $ \(t1 `InnerJoin` t2) -> do
on $ t1 ^. Table1Column1 ==. t2 ^. Table2Column1 &&.
     t1 ^. Table1Column2 ==. t2 ^. Table2Column2
return (t1, t2)


feat: add ability to select dirs of the repository

:tada: This PR is included in version 1.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add the ability to for a user to put in their dietary preferences and prompt them for it on some cases

Allow a user to input their dietary preferences somewhere on their User Profile.
In the case that a User is RSVPing for an event that has had the "food will be provided" option checked and they have not given dietary preferences prompt them to enter them.

 Add the ability to for a user to put in their dietary preferences and prompt them for it on some cases #758


Use karma-browserify for tests. Add browser test coverage reporter.

Hey.
So I started writing some more browser tests. But before I did that, I wanted to make the test runner to compile browserify by itself, and also add some coverage checking (to make it easier to write tests).
This PR does the following:

Modifies the scripts in package.json to not compile with browserify directly.
Modifies the clean script to delete the coverage folder (not sure if this is preferred over adding it to .*ignore)
Add some new dependencies.
Make sure karma does both coverage and browserify.
Also added stderr from the child process that runs karma.

I have some follow up questions about how we should structure the actual test suite, but let's do that in another issue.

Can you use ~ instead of ^ in the versions for the new dependencies?
Don't we still need to remove tests/browser/test-browser.js in the clean script?
nikku/karma-browserify#91 + a new release of karma-browserify should fix the build failure.

Ah. I didn't even notice the test failed. Sorry. And now that I see it I have actually run into the issue before, hehe. Nice work on patching so quickly!

Don't we still need to remove tests/browser/test-browser.js in the clean script?
No. test-browser.js will never get built with this change. It was the bundled version of test.js and now if we use karma-browserify it will get built in a temp file in the os temp dir (as you probably got from the referenced issue).

The reason I added rm coverage instead is because there is a js file in there that messes with the linting. Also, we don't want that in the repo or npm. So there's the question: Should we add it to .npmignore .eslintignore and .gitignore?
Updated dependency syntax in a new commit. Will probably fail testing again though, so better leave this open pending the issue above?
For reference: The reason I locked karma-coverage to version 0.2.6 is this issue: karma-runner/karma-coverage#123

Yeah, let's ignore it and get rid of the clean script.  We shouldn't need to put it in .eslintignore because we explicitly specify which paths are to be linted, right?

Added coverage to .npmignore (it was already in .gitignore). Removed clean script.
You are right, we do specify paths explicitly for eslint, so no worries there.

karma-browserify 3.0.1 is out, if you can explicitly upgrade to that version then I think we're good to go here.

done!

:+1: thanks!


Cursor resets to legacy on text entry

Environment
Windows build number: Microsoft Windows v.10.0.18363.778
Windows Terminal version: PowerShell v.1.18362.752
Note: Changes have been made to the background color, font size, and scroll-forward has been disabled.
Steps to reproduce

Change the cursor to a non-legacy one
Type something

Expected behavior
Preferred cursor remains persistent.
Actual behavior
Non-legacy cursor remains persistent on console (re)launch only. As soon as any characters are typed in the console, the cursor is reset to the default, legacy version.

This is with the traditional console host -- not Windows Terminal -- right?

It's the regular PowerShell, if that's what you mean.

Excellent. /dup #4124
This should be out with one of the upcoming windows releases. (Sorry, we can only service the inbox console so quickly. :smile:)

Now to find out what 'inbox console' means... 🤔

Oh, sorry about that.
So! Powershell and CMD both use the same console window code to get their text onto the screen. It's called "conhost" (short for "console host", becuase it "hosts" the console APIs). It lives in "c:\windows\system32\conhost.exe".
Inbox -- i assume you know what that one means, but I'm on a roll :smile: -- is just "things that come with the Windows OS". They get updated through Windows Update or the windows install ISO.
This repository contains the source code for the console host and the new Windows Terminal (which is like the console host, but it has tabs and a lot more customization!), so we usually handle all the inbox console complaints here too. Same code, same bugs!

The "inbox console" is conhost.exe, the terminal window that you're probably used to seeing in Windows. This is the window that appears whenever you launch cmd.exe or powershell.exe directly.
conhost is this window:

Oh, wow, totally forgot about this thread. 🤔
Yeah spent a good 30 minutes figuring out the difference between a console, terminal, and shell. 😆


DelegateParameter infer methods to find root parameters, instruments, and channels

This PR adds methods to recursively search a chain of DelegateParameters with the aim of finding the root parameter. This also allows users to infer the InstrumentModule or Instrument that a chain of DelegateParameters stems from.
These methods also allow the user to customize which attributes of a Parameter link to other parameters.

While writing tests, I realized there is a possibility for a DelegateParameter "loop" which would cause both get_root_parameter and get_parameter_chain to recurse infinitely.
There are certainly ways I could stop this, but I'm not sure the case is likely enough to bother with at this time. I invite other perspectives on the matter.

There are certainly ways I could stop this, but I'm not sure the case is likely enough to bother with at this time. I invite other perspectives on the matter.

I think it’d be useful to prevent the loops or at least error out when loops are discovered because that will be a welcoming and helpful user experience, so I’d encourage to do it

There are certainly ways I could stop this, but I'm not sure the case is likely enough to bother with at this time. I invite other perspectives on the matter.

I think it’d be useful to prevent the loops or at least error out when loops are discovered because that will be a welcoming and helpful user experience, so I’d encourage to do it


[RFE] Enable tcp_wrapper support in Flatcar, especially with sshd but possibly other daemons as well.

Current situation
Setting up a high secure environment it is good practice to always deploy dual layered security. With respect to sshd, where you need to have the daemon running to be able to do some maintenance, you want both a firewall to prevent access of unwanted connections, but if the firewall fails on you for whatever reason, you want a second layer of defense, and tcp_wrappers is in general a good second layer to allow only connections from specific host names.
Implementing hosts.allow files in a Flatcar linux install resulted in no added security and checking with ldd we find that indeed tcp_wrapper support is not enabled.
node-1 ~ # ldd /usr/sbin/sshd
linux-vdso.so.1 (0x00007ffccdfde000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f23652c0000)
libpam.so.0 => /lib64/libpam.so.0 (0x00007f23652ae000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f2365259000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f236517f000)
libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f2364d43000)
libz.so.1 => /lib64/libz.so.1 (0x00007f2364d29000)
libc.so.6 => /lib64/libc.so.6 (0x00007f2364b46000)
libaudit.so.1 => /lib64/libaudit.so.1 (0x00007f2364b18000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f2364ae9000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f2364ae2000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f2364ad3000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f2364ac1000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2365432000)
libcap-ng.so.0 => /lib64/libcap-ng.so.0 (0x00007f2364ab7000)
On a normal linux install I would expect something like:
ldd /usr/sbin/sshd |grep libwrap
libwrap.so.0 => /lib64/libwrap.so.0 (0x00007f69bbf9f000)
Impact
There is no viable option to create dual layered security on a Flatcar host which results in having to deploy other second layers which are in general more cumbersome to implement, or don't give the added benefit tcp_wrappers gives you.
Ideal future situation
sshd is compiled with tcp_wrapper support and the needed libraries are installed as well.
Implementation options
compile sshd with tcp_wrapper support.
Additional information
Adding this into Flatcar should have minimal impact because there are no hosts.allow or hosts.deny files in /etc at the moment, which results in no rules being in place. When we enable this feature in sshd we can tell everyone that this feature can then be used by just adding the correct files in /etc.

I can also recommend to look into bpf programs if you want to filter a systemd unit: https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#BPFProgram=type:program-path
The benefit is that the kernel will handle this and not the program itself, meaning that even if the program doesn't go through the wrapper lib but uses direct syscalls, it will still work.

eBPF sounds like a good alternative but:
node-1 /etc/systemd/system/sshd.service.d # netstat -anp  |grep ssh
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN      2072/sshd: /usr/sbi
tcp        0      0 <IP_ADDRESS>:22         <IP_ADDRESS>:57408      ESTABLISHED 6421/sshd: jhp [pri
tcp        0     52 <IP_ADDRESS>:22         <IP_ADDRESS>:52202      ESTABLISHED 6180/sshd: jhp [pri
tcp6       0      0 :::22                   :::*                    LISTEN      2072/sshd: /usr/sbi
node-1 /etc/systemd/system/sshd.service.d # systemctl show sshd |grep IPAddress
IPAddressAllow=2a02:b70:0:2:3238::1/128 2a02:b70:0:2:3246::1/128 <IP_ADDRESS>/32 <IP_ADDRESS>/32
Maybe I'm doing something wrong, but it does not seem to work.

I guess you also have to set it for sshd.socket

I have masked sshd.socket and according to the documentation of systemd you should be able to set it on any unit.

Looks like you need to create a global deny and then a specific allow.

So if this is doable with systemd service configuration - i dont think we want to pull in another way of achieving the same result.
Would you agree?

I would agree. I think we can close this again.


Enable setting metadata.ownerReference in RestoreItemAction plugin

Describe the solution you'd like
I'd like to backup/restore resources related to service catalog.
When you create ServiceBinding (CRD from service catalog) there's also created a secret with ownerReference set to that ServiceBinding.
When I restore the secret on another cluster the ownerReference is deleted ( code ).
I'd like to be able to restore the secret with ownerReference set.
I don't want to preserve the old ownerReference because the uid of restored SB is different than from the origin cluster.
I'd like to set the new one, f.e. by plugin. But looking through the code I think it's not possible now because the resetMetadata is after the plugin execution, am I right?
Is it possible to move plugin execution after the metadata reset?
Anything else you would like to add:
Ref: slack thread
One more 'nice to have':
is it possible to pass to plugin not only object with deleted metadata but also original object read from the backup?
If I get object with deleted metadata I have to rely on secret's name (which I think should be equal to SB name) but if I get object from the backup I can get old ownerReference.
Environment:

Ark version (use ark version): master
Kubernetes version (use kubectl version):
Kubernetes installer & version:
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):

is it possible to pass to plugin not only object with deleted metadata but also original object read from the backup?

If we move the execution of the restore item actions first, before resetting metadata, that would do what you need, wouldn't it?

is it possible to pass to plugin not only object with deleted metadata but also original object read from the backup?

If we move the execution of the restore item actions first, before resetting metadata, that would do what you need, wouldn't it?

In such case I have access to the original object, but even if I change the uid in ownerReference than it will be deleted in resetMetadata step.

Right. It's times like this when having all the objects in memory at the same time, prior to restoration, would be useful. But Ark isn't currently set up that way, so the options I can think of are:

Run restore item actions, then reset metadata (don't strip ownerReferences)
Reset metadata, then run restore item actions (modify interface as you suggested, to include original object's state too)
Move to a 2-pass system, where pass 1 has Ark read all objects so it can set up proper ordering, and pass 2 is restoring into the cluster. This would allow us to make your plugin generic - find all objects with ownerReferences, add all the owners to the top of the priority list, restore all the owners first, then recreate ownerReferences with the right data (uids). I don't think this idea is fool-proof, however; we currently restore pods before replicasets, for example, so the pods are preexisting and can therefore be adopted by replicasets. This idea would break that (it would create the replicasets first, giving them time to create new pods on their own before Ark can restore the pods from the backup).

Let's keep the discussion going. Hopefully we can come up with a good solution! @heptio/ark-team please weigh in too!

#3 seems like the biggest change, but might also help with things like https://github.com/heptio/ark/issues/314. Both of these issues want to create something additional to the item currently being processed, though this one is creating a new item vs getting another out of the backup.
I think we've briefly discussed a 2 pass system before, but not in detail. I think I'm in favor of brainstorming around it, even if we decide it's not the right direction.

Option 1 can cause deletion of restored object if there will be no plugin for the restore action.
Option 2 will be sufficient for me because I can set the prioryty of restore in config and it will works
Option 3 - the easiest one for users especially if there will be a generic plugin for updating uids. but also in theory there can be a cyclic  dependency.

Before I read @nrb's response I was thinking option 3 would be more flexible in the long term. I agree with what @nrb, we should at least brainstorm it. So, my opinion: either brainstorm option 3 and see if it's worth implementing, or implement option 2 and add option 3 to the backlog for later.

One more thing: option 3 will override restore priority set in config. So in case of option 3 maybe it should be an opt-in?

I know you're busy with 0.10 release but I would like to move it forward.
What do you think about going first with option 2 and leave additional enhancements as a backlog issue?

@ncdc Is it possible to extract part of this issue (modify interface as you suggested, to include original object's state too) and add it to the epic #1088 ?

@mwieczorek yes, however, the ownerReference typically includes the UID of the owning object. As we've seen in #1037, if Ark creates an object and preserves the ownerReference of the original object, including its original UID, the cluster most likely will end up with too many copies of things (see my description in 1037 for more details).
And, as I wrote above, if we do

Restore owner e.g. ReplicaSet
Get UID of owner
Restore pod, setting its ownerReference to ReplicaSet + new UID from 2

it's possible & likely that Kubernetes will have already created a pod as soon as we restore the ReplicaSet.
Have any suggestions?

There is also #612 asking for the backup/restore to be passed to the backup and restore item actions. Not 100% related to this request, but another possible change to the signatures.

@ncdc I fully agree that setting new UID f.e. for pods doesn't make sense. And I don't want to be done by Ark itself.
What I'd like to be able to do is to enable plugins to set new UID. But for now, in plugins I don't have original UID and other metadata from ownerReference (ownerReference is deleted from the object before the plugins are executed).
So what I propose is to only modify the interface of the plugin and additionally pass into it the original object (from which I can get the original ownerReference kind, name, etc).
We can narrow the scope of this issue to only such change or create a new one (let me know what do you think).
Regarding #612 - it also extends the interface but not with the original object but with backup definition itself.

I'll open a new issue for expanding the interface and then I guess we can close this one?

New issue is https://github.com/heptio/ark/issues/1092

Yes, I'll close this one. Thanks.


kconfiglib: Add preprocessor and two warnings

Update Kconfiglib to upstream revision 547fced630611 to get a new
Kconfig preprocessor in, documented in
https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.txt.
The preprocessor allows shell functions to be run directly from Kconfig.
Things like 'default "prefix-$(shell,some-cmd)' and
'depends on (success,some-cmd)' are supported.
The preprocessor might come in handy for Kconfig/DTS integration. I'm
thinking of extending it so that Python functions can be called as well.
There's also two new warnings:

Trying to use an int/hex symbol like a bool symbol in an expression
(where it will always evaluate to n)

Having a 'default' on an int/hex symbol that lies outside an active
'range'.

The parser is more strict now as well (due to dropping some hacks for 3+
year old kernel versions).
Signed-off-by: Ulf Magnusson<EMAIL_ADDRESS>

CC @carlescufi

Codecov Report

Merging #8832 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8832   +/-   ##
=======================================
  Coverage   52.32%   52.32%
=======================================
  Files         195      195
  Lines       24730    24730
  Branches     5140     5140
=======================================
  Hits        12941    12941
  Misses       9715     9715
  Partials     2074     2074

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0450fc...a028ee7. Read the comment docs.


Translate untranslated Strings

Make former untranslated strings translatable and update translations. For reference: Russian, Portuguese and German are complete in this pull request.

Awesome work!


Add new runner api support

This MR adds support for the new user-based Gitlab Runner API, documented here: https://docs.gitlab.com/ee/api/users.html#create-a-runner
It's worth noting that it's in the users.go file even though it's creating a runner, because it's in the /user api space. I could be convinced to move it to the runners.go file pretty easily, but figured the default stance was to align to the upstream API space.
It's also worth noting that there is no /users/runner DELETE api, because it uses the existing APIs within runners.go to delete by ID.
Let me know if you have any questions! I'd appreciate cutting a new version after merging this MR so that I can start working on the tf provider resource :)

@svanharmelen  - This should be ready for review when you have a moment!

@metanerd

I just found your PR, because I am also looking forward to this feature in the terraform provider. To my knowledge working on the tf provider resource should already work with go mod replace. With this message, it is not my intention to put pressure on this or the tf provider PR.

You're right, I could actually compile the library, use it locally, the develop that way. Sander is usually pretty quick on the review though, and between that an Renovate it means it's usually just cleaner to write it cleanly the first time :) . No worries on the pressure on the provider, I know it's a highly anticipated resource!

Will do; thanks @svanharmelen !


"Enhancement: forward refs" actually broke forward refs

After #69 was merged, forward refs are not parsed correctly anymore.
It complains that the type, which is forward referred to, is not defined.
Code:
@frozen_dataclass
class Comment:
    ...
    replies: List['Comment']

Error:
pedantic.exceptions.PedanticTypeCheckException: In dataclass "Comment" in field "replies": An error occurred during type hint checking. Value: [Comment(...)] Annotation: typing.List[ForwardRef('Comment')] Mostly this is caused by an incorrect type annotation. Details: name 'Comment' is not defined

Thanks for reporting!
Unfortunately, I'm not able to reproduce this. The following snippet runs without any problems:
from typing import List

from pedantic import frozen_dataclass, frozen_type_safe_dataclass

if __name__ == '__main__':
    @frozen_type_safe_dataclass
    class Comment:
        replies: List['Comment']

    comment = Comment(replies=[])
    comment = Comment(replies=[Comment(replies=[])])

    @frozen_dataclass
    class Comment:
        replies: List['Comment']

    comment = Comment(replies=[])
    comment.validate_types()
    comment = Comment(replies=[Comment(replies=[])])
    comment.validate_types()

Tested against current master, which is pedantic version 1.13.1 which points exactly to the PR linked by you.

Ah, I missed a crucial part of the stacktrace (which is quite long and confusing).
The problem occured in a copy_with statement, I think.
Could it be the copy_with implementation does not correctly handle ForwardRefs?
type_ = 'Comment', globals_ = None
context = {'Any': typing.Any, 'Callable': typing.Callable, 'Dict': typing.Dict, 'T': ~T, ...}

    def resolve_forward_ref(type_: str, globals_: Dict[str, Any] = None, context: Dict = None) -> Type:
        """
            Resolve a type annotation that is a string.

 Value: {value} Annotation: '
                f'{type_} Mostly this is caused by an incorrect type annotation. Details: {ex} ')
E           pedantic.exceptions.PedanticTypeCheckException: In dataclass "Comment" in field "replies": An error occurred during type hint checking. Value: [Comment(...)] Annotation: typing.List[ForwardRef('Comment')] Mostly this is caused by an incorrect type annotation. Details: name 'Comment' is not defined

.../lib/python3.10/site-packages/pedantic/type_checking_logic/check_types.py:136: PedanticTypeCheckException

Thanks a lot, that's a good hint!
I can reproduce it now:
from typing import List

from pedantic import frozen_type_safe_dataclass

@frozen_type_safe_dataclass
class Comment:
    replies: List['Comment']

if __name__ == '__main__':
    comment = Comment(replies=[])
    comment.copy_with(replies=[Comment(replies=[])])

I'll fix it asap!

Please check out the new version 1.13.2 which should fix your issue: https://pypi.org/project/pedantic/

I have updated the package, but unfortunately no, the issue remains. Same stack trace.

Okay, that's strange. Can you share a minimal example that produces your bug please?

@DasCapschen Please try Pedantic 1.13.3

This fixed the issue :+1:


Not working properly in some browsers

I'm opening this issue because:

[x] I have found a bug

[ ] I want to request a feature

[ ] I have a question

[ ] Other

My Go version is: (paste the output of go version and remember that qrcp requires at least version 1.8 of Go)

My GOPATH is set to: (paste the output of echo $GOPATH)

Some browsers will pop up a download box. At this point the waitgroup count will immediately become 0
The result is the same as #147  . Scan the QR code and the server will close.
Browsers  E.g:
Edge on IOS
Quark on Android

Thanks for reporting. I will try to reproduce the issue on those two browsers. In the meantime, as a workaround, you can use the --keep-alive flag, which will leave the tool running.
Please let me know if that helps, thanks!

Similarly, Brave on iOS behaves exactly the same

Does "leaving the tool running" mean indefinitely or just until the command is broken via ctrl C?

Hello, it will leave it running until CTRL C is pressed!


Deprecation warning for posts in Jekyll 3.1.3

Hi there,
After I installed this plugin, I get the following deprecation warning:
Deprecation: posts.each should be changed to posts.docs.each.
                    Called by /Users/svetlana/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/jekyll-last-modified-at-0.3.4/lib/jekyll-last-modified-at/generator.rb:7:in `block in generate'.

I don't know if it's the same thing, as in issue #37 that has already been reported.

+1

+1
any chance to fix this problem?

+1
who can help me with this problem???

@SummerBuger feel free to check out my lightweight alternative. Using it in production since half a year.

PR #43 fixed this for me.
Explanation: My issue was that Jekyll-sitemap was requiring Jekyll 3.3 and was depending on jekyll-last-modified-at 0.3.4 (which made the deprecated call) and not 1.0.0 (which fixed that I think). In order for Jekyll-sitemap to use 1.0.0 (or in this case, 1.0.1), some dependencies needed to be updated.
I just ran gem update after these latest commits and everything is working great again. No more warning.

Gem is locked to >= 3.7, and GitHub Pages itself runs 3.8. Seems like a fix for this was made anyhow, closing as stale.


Make DefaultHttpClientFactory not depend on MS DI

Seems like all this logic would be nice to use without using the MS DI stuff. The only reason IServiceProvider is injected into it is for some sort of internal testing.
I'd like to just register it with Autofac. I don't use MS DI.
And it's internal. Weird.

See this thread
https://github.com/aspnet/HttpClientFactory/issues/55
In short, we’re not going to do that becuse the entire configuration API is built on it. You can build up your client then resolve the factory for use in autofac. Another approach would be to use the autofac adapter if you want to replace the container. See
https://github.com/aspnet/HttpClientFactory/issues/127 for more details .

You don’t have to copy anything if you don’t want to just pretend like the di container is a configuration API and ignore it. Add the resulting imstance to autofac

Can't. It pulls in too much stuff. Logging, etc.

So just so I'm clear on what is being asked for here. You want the piece of code that rotates handlers and nothing else in a separate package. Is that right?

Yes !
Rotation and cleanup, just give an hook to create a new one when needed based on context (let us pass some parameter or a function to the CreateHttpClient to configure the factory).
One thing is not clear to me, who handles the connection pooling ? HttpClient or MessageHandler ?

Yeah, I don't know how else to put this. This is a trivial library. I should be able to just new up a HttpClientFactory, stick it in a static. Or my own container. Or whatever.
Instead it's basically pegged to a specific DI implementation. I mean, that's a commonly known Bad Thing To Do. I'm not even sure how to further expand on it.

How many features did you remove along the way? I think it’s great you made it fit your needs but I’d love to know what you did anyways.

Pretty much just took DefaultHttpClientFactory and removed the direct usage of IServiceProvider and such. Func<HttpMessageHanderBuilder> became a constructor arg.
Left the IOptions stuff.
No need for the Typed stuff.

@wasabii did you add the possibility to pass parameter to the httpclient/handler factory ?


for...of is an ES6 feature

In issue #9, @lukasbuenger decided to remove all ES6 features from this project. Unfortunately, it seems like there is one ES6 feature remaining in the codebase. for...of was introduced in ES6, and as a result, it's still causing problems for UglifyJS (and presumably, other software that only works with ES5).
ERROR in application.js from UglifyJs
Unexpected token name «of», expected punc «;» [/Users/singingwolfboy/clones/impraise-web/node_modules/immutable-treeutils/index.js:165,0][application.js:215807,17]

Personally, I'm happy to switch to uglify-es, which is compatible with ES6, but since the author specifically made this change to avoid this kind of compatibility conflict, I wanted to raise this issue.

David, does the 1.0.1 work with your UglifyJS config again?
Many thanks for your feedback in advance.

1.0.1 works! Thank you 👍


error : infinite loop while processing mergePaths plugin.

I'm calling svgo with the following option:
--multipass --disable=convertTransform
For this particular .svg the result is a file with all the markup aside from svg element removed:
https://gist.github.com/matbech/0629e2ab460af94f9339

Wow! It's the type of error that shouldn't occur.

Your error is due to transform="translate(.5-89.26)". SVG syntax doesn't allow omitting the separator between numbers in transform functions. Did you use SVG Cleaner or such?

That was most likely the case. Thank you for pointing it out.

Just released v0.6.1 which should work well now.


Write ADR to choose workflow/asynchronous process framework

At least a few parts of this project require asynchronous process/task running, including but not limited to the following:

initial thumbnail generation to run on new layers (likely python, shelling out to gdal)
initial polygon mask generation to run on new layers (likely python, shelling out to gdal)
kicking off a batch spark job that does not need to be interactive (e.g. via spark-submit)

Most of our familiarity as an organization and team in this area is focused on primarily two frameworks:

Amazon Simple Workflow Service (SWF)
Celery

Our experience has been mixed at best with these two frameworks, both of which can require significant work to get much useful done depending on the use-case. Additionally, tools for tracking and visualizing workflows are very much lacking with these frameworks.
Before we decide on a framework there are probably a few frameworks that we should evaluate:

Luigi -- framework written by Spotify and used at a number of businesses, with nice visualization/job-tracking and out-of-the-box integration with tools we need to use (S3, postgres, hadoop/spark)
Airflow -- similar to Luigi, but built by AirBnB

There may be others, but at first glance these seem like possibly the best-suited to our use-case

Other things to consider when evaluating workflow frameworks:

can they version jobs/workflows to enhance blue/green deployments
is there a way to pin a set of jobs to a single worker -- for instance, to prevent having to re-download a file onto multiple workers if a set of jobs uses the same file