Add extraterrestrial and direct spectra of ASTM G173-03 standard spectrum (AM1.5)

[x] Closes #1963
[x] I am familiar with the contributing guidelines
[x] Tests added
[x] Updates entries in docs/sphinx/source/reference for API changes.
[x] Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
[x] New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
[x] Pull request is nearly complete and ready for detailed review.
[x] Maintainer: Appropriate GitHub Labels (including remote-data) and Milestone are assigned to the Pull Request and linked Issue.

Adds the remaining extraterrestrial and direct spectra of the AM1.5 standard spectrum bundled with pvlib.
Currently, only get_am15g allows retrieving of the global spectra.
The proposed approach adds a new function pvlib.spectrum.get_ASTM_G173 which returns a pandas.DataFrame. API designed similarly to get_am15g, although numpy.interp has been used instead of the scipy counterpart since the latter is deprecated.
I've changed get_am15g to only be a wrapper of this new addition (so we can save the space by removing its data source file).
Feel free to point any suggestions, tips or discuss the solutions.
Docs

https://pvlib-python--2039.org.readthedocs.build/en/2039/reference/generated/pvlib.spectrum.get_reference_spectrum.html
https://pvlib-python--2039.org.readthedocs.build/en/2039/gallery/spectrum/plot_standard_ASTM_G173-03.html
https://pvlib-python--2039.org.readthedocs.build/en/2039/reference/effects_on_pv_system_output/spectrum.html

Shall we take some action here with #1056? I propose adding a subfolder under pvlib/data, for example called spectra or reference_spectra, only with the ASTM file currently.

A couple comments:

Should we consider simply providing (pseudo) constant pandas.DataFrames with the tabulated G 173-03 spectra? Decoupling the data structure from from the interpolation functionality seems like a more extensible approach. A user might have a whole dataframe of measured spectra as a time series that they would like to bulk calculate spectral correction factors for and thus need to interpolate the whole set. One would think the same (separate) interpolating function could also be used for that purpose too. (A detail here is that there is usually a higher level "inner product" function that is called that itself would make use of a separate interpolating function, and the wavelength partition is usually the "common refinement" of a spectrum and a spectral responsitivity, both of which need to be interpolated.)

The extraterrestrial spectrum used in G 173-03 is discussed there (see excerpt below), but it does not actually appear in the tables there with the other two spectra. It might be a good idea to verify and document the source/"chain of custody" of this spectrum (e.g., NREL).

In order to provide spectral data with a uniform spectral
step size and improved spectral resolution, the AM0 spectrum
used in conjunction with SMARTS to generate the terrestrial
spectrum is slightly different from the ASTM extraterrestrial
spectrum, ASTM E 490. Because ASTM E 490 and SMARTS
both use the Kurucz data, the SMARTS and E 490 spectra are
in good agreement though they do not have the same spectral
interval step sizes, spectral interval centers, or spectral resolution.

Should we consider simply providing (pseudo) constant pandas.DataFrames with the tabulated G 173-03 spectra?

Good idea, I think.

@cwhanse and @markcampanelli
I'm not sure if I understand correctly, are you proposing complete removal of the interpolation? I don't have an issue with that, but shall we have the interpolator somewhere else (another PR) or let the user make it?

@echedey-ls I was only suggesting to separate out the two (keeping the interpolator functionality, but separate). Due to issues like supporting interpolation over pandas datastructures like Series and DataFrame, one could argue that interpolation operations should indeed have their own wrapper around numpy.interp. (I could see the other side of this argument that the wrapper does more bloat than good and locks in certain non-universal "opinions" about how to do the interpolation, but users can always role their own.) Supposing we keep the (separate) interpolation, then I would try to "harmonize" the interface with the existing interp function for IAMs in https://github.com/pvlib/pvlib-python/blob/main/pvlib/iam.py#L423 (and other existing interpolators in the code, perhaps?).
Also, I was thinking of just defining the (pseudo) constant dataframe of "raw" standard spectra as something like
SPECTRA_G173_03 = pd.read_csv(
        path_to_csv_file,
        header=1,  # expect first line of description, then column names
        index_col=0,  # first column is "wavelength"
        dtype=float,
    )

However, since Python does not have constant types, the user (or a bug somewhere else in the pvlib library!) could conceivably alter this value. This creates the argument that this "constant" should still be wrapped in a "getter". (It also saves a little overhead during package initialization in which the dataframe would not be created unless actually called for.)

Would we define a separate module-level constant for each standard?  That seems like asking for clutter accumulation over time.  If not, then we will need a getter function anyway, in which case I don't see much benefit in having a redundant module-level (pseudo-)constant.
Also, in case it's not clear, the function as currently implemented returns the spectra dataframes in their original form by default; the interpolation to other wavelengths is an opt-in feature.

A user might have a whole dataframe of measured spectra as a time series that they would like to bulk calculate spectral correction factors for and thus need to interpolate the whole set.

FWIW, pvlib.spectrum.calc_spectral_mismatch_field takes care of such interpolation/alignment internally.

The extraterrestrial spectrum used in G 173-03 is discussed there (see excerpt below), but it does not actually appear in the tables there with the other two spectra. It might be a good idea to verify and document the source/"chain of custody" of this spectrum (e.g., NREL).

Interesting!  I suggest we include notes like this in the function docstring somehow.

Would we define a separate module-level constant for each standard? That seems like asking for clutter accumulation over time. If not, then we will need a getter function anyway, in which case I don't see much benefit in having a redundant module-level (pseudo-)constant.

@kandersolar With the big caveat that I don't have a good grasp of downstream use cases, the level of abstraction that feels right to me on this would be a getter with a selector that returns an entire collection of spectra, from which the user then subselects an individual spectrum by column name.
Here is an example showing current and potential future spectra collections, where the user makes their own pseudo constants from a common get_spectra_collection getter:
ASTM_G173_03_SPECTRA = get_spectra_collection(collection="astm_g173_03")
ASTM_G173_03_SPECTRA_EXTRA = ASTM_G173_03_SPECTRA["extraterrestrial"]
ASTM_G173_03_SPECTRA_GLOBAL = ASTM_G173_03_SPECTRA["global"]
ASTM_G173_03_SPECTRA_DIRECT = ASTM_G173_03_SPECTRA["direct"]

ASTM_G173_23_SPECTRA = get_spectra_collection(collection="astm_g173_23")

IEC_60904_3_ED2_SPECTRA = get_spectra_collection(collection="iec_60904_3_ed2")

ANTONS_KEWL_COLLECTION_OF_SPECTRA_FROM_DURAMAT= get_spectra_collection(collection="some_appropriate_name")

Perhaps this could even be eventually extended for retrieving spectra collections from online sources?

Thanks for the input @markcampanelli
I believe this function aligns with that proposal you make in your last comment. There's of course a naming difference, but nothing else.

Perhaps this could even be eventually extended for retrieving spectra collections from online sources?

I like that idea and I'm up to do it in this PR (if the team agrees on it), but if we can narrow down some details, like what kind of file would be retrieved. I would need some example endpoint essentially. Even if we knew it was a CSV, we should know beforehand what separators, decimal comma/point, etc are used.

I like that idea and I'm up to do it in this PR (if the team agrees on it), but if we can narrow down some details, like what kind of file would be retrieved. I would need some example endpoint essentially. Even if we knew it was a CSV, we should know beforehand what separators, decimal comma/point, etc are used.

@echedey-ls I would caution you against letting my suggestions "blow up" the scope of the present PR :). Indeed, I would suggest sticking to just the three ASTM G173-03 spectra as provided by NREL. That said, I think it is worth considering presently how/where this data provider might intersect with, in particular, the iotools package. There is certainly existing precedent for your new data provider to live right in the spectrum package (cf. get_example_spectral_response and get_am15g). I cannot provide much more guidance here, but I think it's worth a check by the maintainers as to if this is a good opportunity to change the provider location to iotools. Or, maybe iotools is reserved for only EXTERNAL data providers with an API (for which the DURAMAT datahub spectra would apparently qualify, but not NREL's spectra, which you apparently have to download manually as a CSV).

Thanks for the in-depth review @kandersolar !! Ready for another one 👀
Should I open an issue with the deprecation, so it's tasked in a milestone?
Docs

https://pvlib-python--2039.org.readthedocs.build/en/2039/reference/generated/pvlib.spectrum.get_reference_spectrum.html
https://pvlib-python--2039.org.readthedocs.build/en/2039/gallery/spectrum/plot_standard_ASTM_G173-03.html
https://pvlib-python--2039.org.readthedocs.build/en/2039/reference/effects_on_pv_system_output/spectrum.html

Thanks for the in-depth review @kandersolar !! Ready for another one 👀
Should I open an issue with the deprecation, so it's tasked in a milestone?
Docs

https://pvlib-python--2039.org.readthedocs.build/en/2039/reference/generated/pvlib.spectrum.get_reference_spectrum.html
https://pvlib-python--2039.org.readthedocs.build/en/2039/gallery/spectrum/plot_standard_ASTM_G173-03.html
https://pvlib-python--2039.org.readthedocs.build/en/2039/reference/effects_on_pv_system_output/spectrum.html

@echedey-ls Even with the new function staying in the spectrum subpackage, I am still a strong proponent of using the function name and signature
pvlib.spectrum.get_spectra_collection(*, collection="ASTM G173-03" )
instead of
pvlib.spectrum.get_reference_spectrum(wavelengths=None, *, standard="ASTM G173-03")
The proposed alternative function name allows future extension to providing non-standards-derived spectra collections without another function name issue/change, and it also reflects that in general a dataframe of (plural) spectra is returned.
Removing the interpolation option keeps a separation of concerns between data provision and data transformation in the lowest level functions. That said, I realize that this creates the need to add an additional interpolation function to fully replace the existing functionality in the deprecated pvlib.spectrum.get_am15g, which already has the interpolation option. Note: the iam subpackage separates out the interpolation functionality here, so my suggestion could be viewed as bringing more consistency to the overall API.
This is an example usage pattern (with subsequent interpolation not shown):
ASTM_G173_03_SPECTRA = get_spectra_collection(collection="astm_g173_03")  # pandas.DataFrame
ASTM_G173_03_SPECTRA_EXTRA = ASTM_G173_03_SPECTRA["extraterrestrial"]  # pandas.Series
ASTM_G173_03_SPECTRA_GLOBAL = ASTM_G173_03_SPECTRA["global"]  # pandas.Series
ASTM_G173_03_SPECTRA_DIRECT = ASTM_G173_03_SPECTRA["direct"]  # pandas.Series

@markcampanelli can you give an example of a spectra collection this function could be expanded to include in the future?  I'm having trouble thinking of anything that would make sense to provide alongside standards-derived spectra except for other standards-derived spectra.
If we're talking about particular datasets (rather than static reference spectra) like the spectra measured at SNL (Data Hub link) or the Spectral On Demand service, I think we should be talking about one or more separate functions, likely in iotools.

Note: the iam subpackage separates out the interpolation functionality here, so my suggestion could be viewed as bringing more consistency to the overall API.

In my view, pvlib.iam.interp is intended to be an IAM model akin to physical or martin_ruiz.  It happens to perform calculations similar to what we're talking about here, but the motivation is rather different, I think.  Its primary purpose isn't to "re-align" an input IAM profile to some set of convenient AOIs, so it's not really analogous to the spectra interpolation here.  My two cents, anyway!

in general a dataframe of (plural) spectra is returned

Good point!  How about get_reference_spectra?

@kandersolar One case would be a collection of test-problem spectra, but I suppose those types of collections could/should live in a different data provider, be it a different method in pvlib.spectrum or in pvlib.iotools. Sticking to just reference spectra, get_reference_spectra sounds good to me.
I suppose I view pvlib.iam.interp as another function that does double duty (which I generally think adds complexity and potentially inefficiency in usage), as it both transforms a dataset into an IAM function and then computes that function on input angles. That said, I see your point about it's different role. Keeping the dual role here, then I think this signature would be best:
pvlib.spectrum.get_reference_spectra(*, standard="ASTM G173-03", wavelengths=None)
(I don't know pvlib's guidance here, but I prefer to make all passed function arguments require identification by keyword, by using * at the beginning of the signature.)

Thanks @markcampanelli for your feedback, I will change the function name 😃
Regarding the proposed signature, although I prefer reading code with the parameter names, I think it's unnecessary for this function. With just two parameters, one that accepts numbers and other that accepts strings, I doubt we will have any trouble reading someone else's code.

So, are you sure you want to include the * in your current function signature? What is it supposed to accomplish?

@markcampanelli sorry for not being clear enough. I'm usually akin to use keyword-only parameters, but I deem using them here unnecessary. One is a list of numbers that will rarely be provided as a list of magic numbers (i.e., it will in a reasonably-named variable) and the other, a mostly-self-explanatory string.

So, are you sure you want to include the * in your current function signature? What is it supposed to accomplish?

@markcampanelli sorry for not being clear enough. I'm usually akin to use keyword-only parameters, but I deem using them here unnecessary. One is a list of numbers that will rarely be provided as a list of magic numbers (i.e., it will in a reasonably-named variable) and the other, a mostly-self-explanatory string.

@echedey-ls I think the disconnect here is that in my view the wavelengths and standard arguments are "equally optional, both with default values", and I typically view ordering requirements as a source of code complexity. (Here, the root complexity is due to the fact that the function is responsible for multiple tasks.) My understanding of your intent is that if one doesn't name the wavelengths argument, then it must come first, and whenever the standard argument is used, then it must be named.
To be entirely fair, my preference for named-only function arguments (with or without default values) can cause issues when injecting functions into, say, numpy functions that are not designed to handle kwargs. (In this case the complexity is recreated from the need for a wrapper function or some other workaround.) IMHO, Python is perhaps too flexible in the way one can construct function signatures: https://discuss.python.org/t/confusion-regarding-a-rule-in-the-zen-of-python/15927.

That's a fair point.
Anyone wanna weigh in their opinion too? Rn I'm unsure of what to do.

@markcampanelli my apologies for being so stubborn. I think I was confused (I don't remember exactly what came to my mind when I engaged in that conversation earlier). I've re-read that conversation and you are completely right, that keyword only asterisk does not serve any purpose. I've removed it. Thanks for bearing with me.

@echedey-ls No need to for you to apologize or feel silly. Like I said, I think Python would have been better off just making all function/methods arguments keyword only. I find the resulting code considerably easier to read because it becomes self-documenting, and it’s harder to break an interface if/when it grows, inc. if a default value is added.
Going the other way, I appreciate you bearing with my raving on about complexity and concerns about API consistency 🙂.

Thanks @echedey-ls for the PR and @markcampanelli for the reviews!


iPhone SE 3 (iOS 16.6) Installation failed

Previously it worked on version v8.0.8.

I have already restarted many times.


Drawable thumb

Hello!
Great job!
Is it possible to stylize the thumb via drawable with selector?

#95


frontend changes in memory game

PR Description 📜
Please include summary related to the issue you have fixed and describe your PR in brief over here by specifying the issue number on which you were working below
Fixes #467
I have made some minor changes in the CSS file of the Memory Game. My main idea was to make it eye catchy.

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[ ✅] I have performed a self-review of my own code or work.
[ ] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[  ✅] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[ ] I have added screenshot for website preview in assets/images
[ ] I have added entries for my game in main README.md
[ ] I have added README.md in my folder
[ ] I have added working video of the game in README.md (optional)
[ ✅] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

Thank you  @ritika1408 , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀


Allow enabling embedded Swift without WMO when not generating SIL

This allows modes like -index-file to work the same way they do when not using embedded Swift

https://github.com/apple/swift-driver/pull/1513
@swift-ci please build toolchain macOS

I'll defer to @kubamracek about correctness, but I'm very thankful to see improvements on this front :)

Yeah, this is definitely one part, thanks @owenv 🙇! Mind cherry-picking it too?

Dec 23, 2023

:O

@bnbarham yeah, I wrote this over the holiday and then forgot about it for a bit. Let's make sure everything is still working after a rebase
@swift-ci please smoke test

@swift-ci please smoke test Linux

@swift-ci please smoke test macOS

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test


Updated rule debt values

I just updated too high or too low debt values.

Should detekt output the estimated debt in the report and/or include the debt values in the documentation? What do you think?

Thats a great idea! :+1: :+1:

Ok.
Can something be done to resolve the exception in the build besides waiting for a new RC? The detekt.sample.extensions references the old Issue in the old .jar.

@mauin commited a migration gradle task for that I think ...

I don't think I ever built that task for making that work. But the steps you explained should do the trick, yes.

Ok. I did a gradle build on the detekt.sample.extensions project.
I only get detekt-sample-extensions-1.0.0.RC6-3 and detekt-sample-extensions-1.0.0.RC6-3-sources. What did I do wrong?

│  │     └─ it should load the sample provider ✘ java.lang.IllegalArgumentException: Parameter specified as non-null is null: method io.gitlab.arturbosch.detekt.api.Issue., parameter debt
This test case is still broken :/

@arturbosch I rebased after the new RC. I have no idea how to fix it.

Merged manually.


Add Text Component Page

Added page for Text core component.
Snippet of Text Page

This looks great!
Just one request:  I'd also like to see an example of nested Text, illustrating how you can set different styles (eg first do italicized, then do bold in the nested Text, or something along those lines).

This looks great!
Just one request:  I'd also like to see an example of nested Text, illustrating how you can set different styles (eg first do italicized, then do bold in the nested Text, or something along those lines).

This looks great!
Just one request: I'd also like to see an example of nested Text, illustrating how you can set different styles (eg first do italicized, then do bold in the nested Text, or something along those lines).

@kmelmon Nested Text example now added!

This looks great!
Just one request: I'd also like to see an example of nested Text, illustrating how you can set different styles (eg first do italicized, then do bold in the nested Text, or something along those lines).

@kmelmon Nested Text example now added!


[WIP] input methods: use GTK3_IM_MODULE_FILE

Work in progress, please do not merge yet.
This is an implementation of #14019 fix.
@ttuegel Could you review this PR code?
A few personal concerns:

Using a systemd unit to generate the cache files doesn't feel right, is there a better way to do that? solved in the second commit
Having an option (i18n.inputMethod.package) just to pass the input method package doesn't feel right. solved by setting internal = true

Note:

Testing this PR will trigger a quite long rebuild of the gtk+3 package.
I add commits to keep track of progress, all commits will be merged into one before merging.

If gtk-query-immodules-* doesn't access anything but GTK_PATH then you can run it in a package. Then link results from the package into /etc. I have used this approach for example here.

@abbradar Thanks, I added a commit to remove the systemd unit and use mkDerivation instead. It looks cleaner now.

Nice! LGTM, I'll wait for someone else with more GTK expertise to review it but if I understand everything correctly, this should work. Also we may want to port this to stable although it's somewhat mass-rebuildy. @domenkozar, what do you think?

Oh, one more thing -- don't sweat over package option, I believe it's our standard way to do this -- just grep for internal = true.

Looking at previous discussions on the topic (#2450), I think that @lethalman might be interested by this PR.

@ericsagnes Would you mind patching for the other environment variables GTK 3 reads, the same way you have done for GTK_IM_MODULE_FILE? It needs to be done eventually, and I would hate to incur multiple mass rebuilds.

@ttuegel No problems, looking at #5053 I only see GTK_DATA_PREFIX. Are there any other variables that should be patched?

We really need to patch every call to g_getenv("GTK_*") because GTK makes no effort to determine if the listed paths are for GTK 2 or GTK 3.

@ttuegel Result of a quick grep in gtk3 source:

GTK_THEME
GTK_DEBUG
GTK_OVERLAY_SCROLLING
GTK_CSS_DEBUG
GTK_DATA_PREFIX
GTK_EXE_PREFIX
GTK_PATH
GTK_STYLE_CONTEXT_WARNING
GTK_IM_MODULE
GTK_MODULES
GTK_TEST_TOUCHSCREEN
GTK_INSPECTOR_DISPLAY
GTK_WIDGET_ASSERT_COMPONENTS
GTK_CSD

Is it fine to patch all these?

Yes, that sounds good!

From cursory glance, some of these may be unnecessary (for example, GTK_DEBUG or GTK_TEST_TOUCHSCREEN). I'm not sure, however.

Tested using nixos-rebuild build-vm with a configuration.nix below:
https://gist.github.com/taku0/f988cd6c0f1151f8ee89bc7aa24bdcfb
I got a strange behavior: if GTK_IM_MODULE_FILE is defined and ~/.uim.d/customs/custom-* does not exist, uim crashes with the following message:
Error: in scm_symbol_value: unbound variable: generic-commit-key?

The error does not occur with master branch (GTK_IM_MODULE_FILE is not defined) or if ~/.uim.d/customs/custom-* are created with uim-pref-gtk. The error occurs again if ~/.uim.d/customs/custom-* are removed. This could be a bug of uim or Anthy. I will investigate it later if possible.

@ttuegel I added a commit patching all the variables cited in the previous comment.
gtk+3 is compiling fine, but I don't know how to check that all the new GTK3* variables are working correctly.

@ericbmerritt Very cool! Maybe we want to try upstreaming this (of course, out of scope of this PR)?

Ouch, sorry, auto-completion. I meant @ericsagnes

@ericsagnes I agree now that the variables which are testing for NULLity should not have GTK3_ variants. It breaks the logic of these debug options, I think. Although, I'm not sure anything is harmed.

@abbradar Good suggestion, I filed a ticket upstream.

I agree now that the variables which are testing for NULLity should not have GTK3_ variants. It breaks the logic of these debug options, I think. Although, I'm not sure anything is harmed.

@ttuegel I can revert that if you think it is better, should I?

@ericbmerritt Thanks! Let's see what upstream thinks.
From another look (brief, though) those variables can be actually useful to have GTK3_ variants:

GTK_IM_MODULE_FILE
GTK_THEME
GTK_DATA_PREFIX
GTK_PATH
GTK_IM_MODULE

Notice that I haven't included GTK_EXE_PREFIX because it's visible in the patch that GTK uses it with /gtk-3.0 postfix, so it should be fine even when co-used with GTK2. Maybe I'm missing something, though.
Others are not really important and/or mostly for debugging, I think.
Notice that I don't have any knowledge on GTK topics and this is just me reading the patch and concluding what a variable does from its name!

From another look (brief, though) those variables can be actually useful to have GTK3_ variants:
GTK_IM_MODULE_FILE
GTK_THEME
GTK_DATA_PREFIX
GTK_PATH
GTK_IM_MODULE

Notice that I haven't included GTK_EXE_PREFIX because it's visible in the patch that GTK uses it with /gtk-3.0 postfix, so it should be fine even when co-used with GTK2. Maybe I'm missing something, though.

I think the problem with GTK_EXE_PREFIX as things stand now is that it must be shared with GTK 2. I would prefer to have separate variables; in worst-case scenario, they always take the same value. Otherwise, I agree with you list of variables that need separate prefixes. Thanks for looking into this!

@ttuegel Hm, indeed, this would be more convenient (allows us to separate GTK2 and 3 configuration, enable their support separately etc). So I agree, let's patch _EXE_PREFIX too.

@abbradar I added a commit to remove the patches of the following variables:

GTK_DEBUG
GTK_OVERLAY_SCROLLING
GTK_CSS_DEBUG
GTK_STYLE_CONTEXT_WARNING
GTK_TEST_TOUCHSCREEN
GTK_INSPECTOR_DISPLAY
GTK_WIDGET_ASSERT_COMPONENTS
GTK_CSD

So now, the only patched variables are:

GTK_IM_MODULE_FILE
GTK_THEME
GTK_DATA_PREFIX
GTK_PATH
GTK_IM_MODULE
GTK_EXE_PREFIX

Thank you! This looks ready to me now.
Nikolay.

This is a mass rebuild I think, so I'll cherry-pick this to staging -- is this okay? I also think this should be ported to stable -- how do we usually proceed with mass-rebuildy patches to release? cc @domenkozar

I arranged the commits so the PR is ready to merge.
@taku0 Have you found the cause of the uim crash? I tested the PR with uim on a test machine and couldn't reproduce the issue.

@abbradar we can eventually backport to 16.03, there's no staging for it. But for mass rebuilds, it should first be committed to master and tested for a few weeks. Then open a PR to backport to stable with arguments why it's feasible to do so.

@domenkozar Got it, thanks
Let's wait for @taku0's response to ensure it isn't this patch's fault and merge.

It crashes when I try to input to an text field of uim-pref-gtk or uim-pref-gtk3 but not crashes on firefox-bin or emacs. So it would be fine for daily use.

@taku0 I would like to check if the same happens on my test machine, can you tell which text field of uim-gtk-pref and the steps to reproduce the crash?

Download https://gist.github.com/taku0/f988cd6c0f1151f8ee89bc7aa24bdcfb to a temporary directory.
git fetch --force upstream pull/14417/head:pull_14417 in nixpkgs directory.
git checkout pull_14417 -b tmp
git rebase master (optional)
nixos-rebuild build-vm -I nixpkgs=~/src/nixpkgs -I nixos-config=./configuration.nix where ~/src/nixpkgs and ./configuration.nix should be adjusted.
./result/bin/run-nixos-vm
Login as alice with password alice.
(Ignore KDE error message)
Open konsole from start menu.
uim-pref-gtk
Open “SKK 辞書” (SKK dictionaries) group.
Focus “個人辞書ファイル” (Personal dictionary file) text field.
Ensure Anthy is enabled (leftmost button on the uim toolbar)
Switch Anthy input mode (2nd button from the left on the uim toolbar) to “ひらがな” mode.
Type some words into the text field, e.g. ka.
Error messages is dumped to the konsole.

Seeing the upstream reaction on this patch, I wonder why won't we just patch our default paths into GTK instead? So it would expect IM cache in /etc/gtk-*/immodules.cache, have data prefix /var/lib/current-system/sw etc. by default.  What would the disadvantages be?

What would the disadvantages be?

GTK applications won't work outside NixOS. I don't have a problem with that; given upstream's obstructionist attitude, we should drop all GTK packages anyway. That's not going to happen, so I would say: go with the patch we already have.

GTK applications won't work outside NixOS.

A-ha, indeed: I haven't considered that. Thanks for the clarification!

@taku0 Thanks for the detailed steps! I could reproduce the errors in the vm, but it is working fine on my physical test machine.
It might be related to the vm, do you have a way to test on a physical machine to see if it is working any better there?

I will try to test on this weekend.

Update from the upstream bug report:

GTK_* environment variables shouldn't be used and some of them are deprecated (GTK_IM_MODULE_FILE and GTK_THEME)
Instead of patching environment variables, they prefer patches to fix default behavior of gtk; So it is very unlikely they will use this solution.

From the bug report:

I would be much more interested in patches that fix cache file locations like that or just make gtk work without the cache than I am in adding extra environment variables as a 'quick fix'.
The long term vision for this is that we want to move away from loading modules for these things altogether. GTK+ should just use the preferred input method framework of the platform it is running on (for Linux, that would be ibus), not let the use switch between dozens of unmaintained and half-working alternatives.

So shall we merge the PR an keep this patch until upstream fixes the default functionality or try a different approach?

I say let's merge -- while upstream fixes this, we want to have a working solution for our users. I don't think we as distribution maintainers need a better fix (of course, we as users may want one, but that's another story). I also propose to move discussion about @taku0's problem to another issue. If noone objects, I'll hit the button in several days (feel free to beat me to it, of course!).

Thank you, this PR fixed my UIM. :)

...and broke pasystray. Backtrace shows that pasystray's  GTK+3 loads UIM plugin for GTK+2 and gets SIGABRT later. We need to test upgrade path for other GTK+3 apps before merging this.

@astsmtl I tried  pasystray with this PR and couldn't reproduce your issue.
I checked pasystray source to see if it was using any of the variables patched and couldn't find any reference.
Is there any special action that trigger the bug?

@astsmtl Wild guess -- maybe you use pasystray from nix-env without having it rebuilt? This patch would break all such software that uses old (unpatched) GTK+3.

or try a different approach?

Another option is reverting a91161aa57f7cfe447a551bafc2d7c6c6bbbe44b partially for gtkPlugins as ttuegel  suggests (disclaimer: I'm the author of gtkPlugins: #3275), though I'm not against GTK3_IM_MODULE_FILE.

Of course it was pasystray from my profile. But is this a good excuse for breaking? Maybe we can also patch GTK+2 to use GTK2_XXX variables?

Of course it was pasystray from my profile. But is this a good excuse for breaking?

This is a fair point.
One way we could fix this issue without breaking existing packages is to patch GTK 2 and 3 to search NIX_PROFILES for an immodules.cache file. I patched Qt 5 this way for its plugins. We could have GTK search each profile for a file $PROFILE/etc/gtk-{2,3}.0/immodules.cache. This simultaneously solves the backwards-compatibility problem, it allows users to override the system immodules.cache, and it works on non-NixOS systems.

Ideally, the following three GTK+ versions must match:

A. the version used to build the application
B. the version used to build the input method (or other plugins)
C. the version used at runtime

The GTK3_IM_MODULE_FILE approach fixes B system-wide, so A-B and B-C may be different.
The gtkPlugins approach fixes B and C system-wide, so A-B and A-C may be different.
The wrapping-all-GTK+-applications approach suggested by @lethalman on #2450 fixes none of them, so all versions match, but requires to wrap all existing and future GTK+ applications. To make matters worse, if a non-privilege user installs an input method, it will effective only for applications installed by that user but not for system packages.
This is an unsolved problem of Nix.

@ericsagnes
It fails on my physical x86_64 machine.
uim-pref-gtk: fails
uim-pref-gtk3: fails
uim-pref-qt4: works
firefox: works
emacs: works
inkscape: works

One way we could fix this issue without breaking existing packages is to patch GTK 2 and 3 to search NIX_PROFILES for an immodules.cache file. I patched Qt 5 this way for its plugins. We could have GTK search each profile for a file $PROFILE/etc/gtk-{2,3}.0/immodules.cache. This simultaneously solves the backwards-compatibility problem, it allows users to override the system immodules.cache, and it works on non-NixOS systems.

@ttuegel That sounds like a cleaner approach, I made an implementation here. It is working fine with gedit (gtk3) and geany (gtk2). It trigger long mass rebuilds.
This fixes #14019 but does nothing for #5053.
Should I make a new PR with the NIX_PROFILES version and close this one?
@taku0 thanks for testing, I have no idea about what can be the cause as other gtk apps are working. Does the NIX_PROFILES version work any better?

Should I make a new PR with the NIX_PROFILES version and close this one?

Yes, I think that would be preferable.

closing in favor of #14568.


Vector graphics mode and infinite canvas

For making notes it would be convenient to have an infinite canvas, so user could use as much space they want. If we don't want to sacrifice quality of drawing, the drawing must be represented as piece of vector graphics.

UX similar to https://github.com/ARK-Builders/ARK-Retouch/pull/83 should be implemented.


HDDS-1952. Disable TestMiniChaosOzoneCluster in integration.sh

What changes were proposed in this pull request?
Disable TestMiniChaosOzoneCluster in integration test (integration.sh) run by CI and nightly builds, since it always crashes after running for a long time.
It can still be run manually.
https://issues.apache.org/jira/browse/HDDS-1952

/label ozone

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
110
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

0
shelldocs
0
Shelldocs was not available.

+1
@author
0
The patch does not contain any @author tags.

-1
test4tests
0
The patch doesn't appear to include any new or modified tests.  Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1
mvninstall
757
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
1081
branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1
mvninstall
688
the patch passed

+1
mvnsite
0
the patch passed

+1
shellcheck
1
There were no new shellcheck issues.

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
852
patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1
unit
129
hadoop-hdds in the patch passed.

+1
unit
336
hadoop-ozone in the patch passed.

+1
asflicense
61
The patch does not generate ASF License warnings.

4260

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1284/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1284

Optional Tests
dupname asflicense mvnsite unit shellcheck shelldocs

uname
Linux 020854b254e5 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / e4b538b

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1284/1/testReport/

Max. process+thread count
306 (vs. ulimit of 5500)

modules
C: hadoop-ozone U: hadoop-ozone

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1284/1/console

versions
git=2.7.4 maven=3.3.9 shellcheck=0.4.6

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
41
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

0
shelldocs
0
Shelldocs was not available.

+1
@author
0
The patch does not contain any @author tags.

-1
test4tests
0
The patch doesn't appear to include any new or modified tests.  Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1
mvninstall
640
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
783
branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1
mvninstall
574
the patch passed

+1
mvnsite
0
the patch passed

+1
shellcheck
0
There were no new shellcheck issues.

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
678
patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1
unit
103
hadoop-hdds in the patch passed.

+1
unit
286
hadoop-ozone in the patch passed.

+1
asflicense
50
The patch does not generate ASF License warnings.

3341

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1284/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1284

Optional Tests
dupname asflicense mvnsite unit shellcheck shelldocs

uname
Linux e67b550e84a9 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 454420e

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1284/2/testReport/

Max. process+thread count
447 (vs. ulimit of 5500)

modules
C: hadoop-ozone U: hadoop-ozone

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1284/2/console

versions
git=2.7.4 maven=3.3.9 shellcheck=0.4.6

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Thanks @mukul1987 for reviewing it.  Unfortunately excludes only works in pom.xml, not in the command-line.  The updated patch successfully excluded TestMiniChaosOzoneCluster in the CI run.
As a side-effect, it also ran all tests in hadoop-ozone-filesystem (see #1235 for a more complete fix for HDDS-1916).
Acceptance test failure is unrelated (HDDS-1936).

Thank you @nandakumar131 for reviewing and merging it.


Fix multisignature

After the merge of #2582 we will broke multisignature mechanism. It was working because of signature checking mistake (see #2580).

The perfect fix we are thinking of is to allow users to use WASM smartcontract validators to check transactions.
That requires things such as:

WASM validators
Second queue for multisig transactions

@SamHSmith

Also it's wroth notice that our Queue is broken in terms of multisigs too.
Problem is the following:
We have method check_tx which treats false result of signature check as error and does not allow to add such signatures into the queue.
Mutisig transactions that does not received enough signatures would return false (because such transaction is not complete).
Simplest solution for such problem would be to allow putting transaction into the queue without signature_check (or at ) and pop from the queue transactions that pass `signature_check.
Problem with such approach is that malicious actor can flood queue with invalid transactions.

After meeting with @Arjentix  we have to possible solutions for the problem:

Revert to initial approach where we allow to put transaction into the queue even if it not yet pass signature check condition.
Some pros and cons of the first approach:
+) Simple and straightforward to implement;
+) Using isi for signature check is more transparent then wasm;
-) Such approach could suffer from flooding of transactions that don't pass signature check (they will stay in the queue untill expired).

Introduce 3 variant (Allow, Deny, Skip) logic for signature check where we would reject to add transaction into the queue if it is signed incorrectly and allow to add transaction that still waits for more signatures.
Some pros and cons of the second approach:
+) More protected against flooding, because we can cut off some of invalid transactions, but malicious actor still can flood queue with transactions that are waiting for more transactions;
+) Same advantage about transparency;
-) Much harder to write signature check condition for 3 variant logic;
-) Increasing size of Value even further;

For switching to WASM validators we are still considering all pros and cons.

As addition for method 1 and 2 second queue could be introduced, which will partially solve flooding problem, because transactions that are waiting for more signatures could be stored here. And transactions that pass signature check would be processed without any problem. However second queue still can reach it's full capacity.

I will close this issue in favor of #3088


Udpate chromedriver

Move to 2.21.

:+1: want to publish a patch version so we get it into next appium?

Sure!


Multi-AZ and multi-role setup of workers via Labels and CloudProfile

Summary
Currently server selector labels are set via serverLabels in CloudProfile's spec.providerConfig.machineTypes.
Assuming we have metal CloudProfile in a region where we need to create following Shoots:
- Shoot for kvm compute pod in building block BB123 (az-a)
- Shoot for kvm compute pod in building block BB456 (az-b)
- Shoot for ceph pod in across multiple pods - admin pod, storage pod  (stretched az-a and az-b)

We should consider adding additional means of more specific selection of nodes via shoot spec for the respective worker pool:
    - cri:
        name: containerd
      machine:
        image:
          name: gardenlinux
          version: "1592.1"
        type: some-machine
      maxSurge: 1
      maxUnavailable: 0
      maximum: 3
      minimum: 3
      name: worker
      zones:
      - us-east-1a
      providerConfig:  <<<<
        serverLabels:
          type: bb123

@afritzler Do you think we should add the additional selection labels to the InfrastructureConfig similar to #56?

How about we put this in the provider section of machines: https://github.com/gardener/gardener/blob/master/example/90-shoot.yaml#L46

If I remember correctly, that section is intended for the os-extensions. That's the very reason we moved the extra ignition stuff into the infrastructure config.

Sorry my bad. I meant that providerConfig here: https://github.com/gardener/gardener/blob/master/example/90-shoot.yaml#L65


recommended proxy rotation to avoid getting blacklisted?

I know this is a bit out of scope but just thought I'd ask to see if maybe you're already aware of something that works well for this purpose
basically i'm going to be doing a lot of queries from a single server with static ip and want to avoid getting blacklisted. Can you recommend a service and a package (in python I imagine) that would wrap your kijiji-repost-headless well and easily?
thanks!!

My favorite part is when kijiji gets sick and tired of all these people abusing their API and either shuts it down for everyone or tightens posting restrictions such that the API becomes almost completely worthless to non-corporate users.

What you're looking for is not in the best interest for the future success and continuation of this project. I do not support using proxies to get around Kijiji blacklists.

I think you guys may have assumed I was planning on hammering and being malicious which is as far from reality as it could be. I was mostly interested in learning something new and thinking ahead when I offer to let my friends and family use this tool I'm building. I'm not aware of how strict they are when it comes to posting with this kind of script but I just figured it would be nice to make it a little harder for them to detect this behaviour. I never intended to do stupid things like hammer or post/query more than is sensible.
thanks for the continued good work

I'm still trying to figure out what's in it for them to allow access via an API. With more and more people wanting to use it I can only imagine posting limits being reduced and more restrictions being applied.
Convince me otherwise?

the answer to that is simple... there is no API. This package IS the API.. and the number of people using this or other posting services must be a grain in the sand in their overall usage

I do not support using proxies for this project, and do not wish to support proxies. If you wish to develop this feature, please do so on your own fork.


Switch to compare x with rest on errorMargin check

Looks like it's better to check x with rest value, not previous.
For example if errorMargin becomes a user parameter, there will situations with small velocity and small dx values but x is far from the rest.

Also looks like it's better  for spring animation (with check added above) to check velocity
Math.abs(newV) < errorMargin but not an acceleration Math.abs(newV - v) < errorMargin, and it allows to increase errorMargin Look at second commit

PS: Without ability to write custom stepper or set errorMargin as spring parameter there are a lot of small visually bad animation movements.

The logic here is tricky. Does this glitch with animations whose destination is 0? What about the edge-cases of TransitionMotion where you're adding such an item, etc. (this is super vague but can you see what I mean?)
Also, how relevant is this if we implement custom error margin?

Destination value can be zero or any other value, this part Math.abs(newX - destX) contains delta not destination itself so it can be any numeric value.

It's more easy to user think in terms like - if velocity is small enough and distance to destination is small we can stop animation instead of current if acceleration is small and velocity is small we can stop animation

For a big errorMargin master solution could stop animation at first step even if destination value is far far away from value because of acceleration and velocity is less then errorMargin.
Solution proposed above will not stop.

I've just made a search over github and fast looking into the code gives
Looks like facebook rebound use the same approach as above
Looks like JNWSpringAnimation same approach use the same approach (be sure that proposedValue - previousValue is Velocity) or similar

I'm not an expert in edge-cases of TransitionMotion so can't say nothing about, need to read more.

You know what, this is probably right. If TransitionMotion screws up after this then we should fix that instead.
Sorry for the delay! Thanks =).

Thank you for this library!


Help using TriggerDagRunOperator

I running into problems use the TriggerDagRunOperator:
def foo(context, dag_run_obj):
    #What goes here?
    #create a DagRun?
    pass

dag = DAG(dag_id='test_short_circuit_novo', default_args=args, schedule_interval=None)

task = BashOperator(
    task_id='runme',
    bash_command='echo run_me',
    dag=dag)

trigger = TriggerDagRunOperator(task_id='trigger_other_dag', dag_id="test_trigger", python_callable=foo, dag=dag)
task.set_downstream(trigger)

    trigger = TriggerDagRunOperator(task_id='trigger_other_dag', dag_id="test_trigger", python_callable=foo)
  File "/home/pedro/workspace/mdm-source/env/local/lib/python2.7/site-packages/airflow/utils.py", line 472, in wrapper
    result = func(*args, **kwargs)
TypeError: __init__() takes at least 3 arguments (2 given)

Does anyone can write a sample code for the def foo(context, dag_run_obj): function?

Something is wrong in the documentation:
From documentation:
class airflow.operators.TriggerDagRunOperator(dag_id, python_callable, *args, **kwargs)
dag_id (str) – the dag_id to trigger
From test case:
        t = operators.TriggerDagRunOperator(
            task_id='test_trigger_dagrun',
            trigger_dag_id='example_bash_operator',
            python_callable=trigga,
            dag=self.dag)

Hi @pedrorjbr,
Great question and it's very confusing, I agree. This is screaming for better documentation.
I put together an example to help you. There are some gotcha's however that tripped me up:

Make sure you run everything on UTC -- Airflow does not handle non-UTC dates in a clear way at all and in fact caused me scratch my head as I saw an 8 hour delay in my triggered dag_runs actually executing.
When you use the TriggerDagRunOperator, there are 2 DAGs being executed: the Controller and the Target. Both DAGs must be enabled. The Target DAG should have a schedule_interval=None

Illustrative Example:
Step 1 : Define your Target DAG with schedule_interval=None. DAG_RUNs of this target will be created and executed every time that your trigger fires.
from airflow.operators import *
from airflow.models import DAG
from datetime import date, datetime, time, timedelta

args = {
    'start_date': datetime.now(),
    'owner': 'airflow',
}

dag = DAG(
    dag_id='simple_dummy_dag_v1',
    default_args=args,
    schedule_interval=None)

run_this = DummyOperator(
    task_id='run_this',
    dag=dag)

Step 2 : Define your Controller DAG with any interval that makes sense for you! It's going to be checking a condition on that interval and triggering the Target DAG whenever the condition is met. In the example below, I am running it every minute and always executing the trigger. The idea of the foo function is to check a condition and then return dag_run_obj if you want to trigger the Target DAG. If your condition is not met, then return None.
from airflow import DAG, utils
from airflow.operators import *
from datetime import date, datetime, time, timedelta

def foo(context, dag_run_obj):
    if True:
        return dag_run_obj

dag = DAG(dag_id='test_trigger_dag_run_for_Sid',
          default_args={"owner" : "me",
                        "start_date":datetime.now()},
          schedule_interval='*/1 * * * *')

trigger = TriggerDagRunOperator(task_id='test_trigger_dagrun',
                                trigger_dag_id="simple_dummy_dag_v1",
                                python_callable=foo,
                                dag=dag

Optional Step 3 : You can also manually trigger your Target DAG using the CLI, perhaps when testing or if you are using some custom automation via trigger_dag simple_dummy_dag_v1
Your DAG_Run table will look like :

You can see 3 types of entries (from top to bottom):

Manually triggered (via CLI) DAG_RUNs of the Target DAG (e.g. run_id : manual__2016-02-15T22:57:49.996114)
Controller triggered DAG_RUNs of the Target DAG (e.g. run_id : trig__2016-02-15T22:26:37.425566)
Controller DAG_RUNs (e.g. run_id : scheduled__2016-02-15T11:15:00)

The documentation sucks. Please do your part and update it as and when questions are answered. I can do this one. Closing for now, but ping me here or on gitter if you need it reopened.

@r39132: Thanks again for the response. I will update the documentation. Do you have any example how can I pass information from a dag to a triggered dag? How I get the information on the triggered dag?

Good question. Reopening questions and I now understand the other issue that you opened. Post the question on gitter - I know @nicktrav is doing something with triggering subdags. That might hold a solution. I'll investigate as well later tonight,

http://pythonhosted.org/airflow/code.html#airflow.operators.TriggerDagRunOperator
If your python_callable returns True, a dagrun is created, you can pass anything picklable (should be small-ish) to the payload object
def foo(context, dagrun):
    if condition_ismet():
        dagrun.payload = {'execution_date': context['execution_date'], 'foo': 'bar'}
        return True

You can refer to the code

Thanks! @mistercrunch
But how can I get the pickable object in the triggered dag?
Is it a property of the DAG or Task? Can you post a sample of code. Sorry, maybe it is a noob question.

I'm trying to put an example together. I can see my "payload" in the DB under the created dag_run, but can't find an example of where it is accessed. It is very unfortunate that it is stored under the dag_run's conf attribute -- makes grepping the code impossible.

I'll have an example for you shortly - just figured it out.

Here you go:
https://github.com/airbnb/airflow/pull/1043
If this meets your needs, close the issue. Else, please ping me on gitter.. email tends to get lots.

Closing. If you have any questions, add them here and ping me on gitter and I can reopen.

@r39132
I am getting in kwargs, and cannot get the conf value...
{u'dag_run': None, u'run_id': None, u'dag': <DAG: test_trigger>, u'conf': <module 'airflow.configuration' from '/usr/local/lib/python2.7/dist-packages/airflow/configuration.pyc'>, u'tables': None, u'task_instance_key_str': u'test_trigger__run_this__20160220', u'END_DATE': '2016-02-20', u'execution_date': datetime.datetime(2016, 2, 20, 12, 45, 36), u'ts': '2016-02-20T12:45:36', u'macros': <module 'airflow.macros' from '/usr/local/lib/python2.7/dist-packages/airflow/macros/init.pyc'>, u'params': {}, u'ti': <TaskInstance: test_trigger.run_this 2016-02-20 12:45:36 [running]>, u'ds_nodash': u'20160220', u'test_mode': False, u'end_date': '2016-02-20', 'templates_dict': None, u'task': <Task(PythonOperator): run_this>, u'task_instance': <TaskInstance: test_trigger.run_this 2016-02-20 12:45:36 [running]>, u'latest_date': '2016-02-20', u'yesterday_ds': '2016-02-19', u'ts_nodash': u'20160220T124536', u'tomorrow_ds': '2016-02-21'}

Let's keep the conversation in a single issue. Let's keep the conversation in #1029 I'll change that title to be the same as this one  'Help using TriggerDagRunOperator'


Generated zarf.yaml reference helm chart rather than manifest

Is your feature request related to a problem? Please describe.
The current npx pepr build generates a helm chart + a manifest (from the templated chart?) for deployment of pepr. The zarf.yaml that is also generated references that manifest. This limits a lot of possible end user config options (labels, resources, affinity, etc). While an end user could make a new zarf.yaml or modify the generated one, it seems preferable that the default would be to use the chart.
Describe the solution you'd like
When I npx pepr build, the zarf.yaml that is generated uses the helm chart. This allows me to import this zarf.yaml other locations and extend with values files.
Describe alternatives you've considered
As mentioned this could be done with a new zarf.yaml referencing the chart.
Additional context
uds-cli heavily leans into helm charts for overriding values in a bundle. This does provide a way that someone could modify certain pepr pieces (for good or bad) so it may require some careful consideration on whether all existing items should still be exposed as helm values.
As an aside, this would also make things like "flavored pepr" closer aligned to the approach we take in uds-core with other flavored items, using values to override the image rather than the build flag.

This probably makes sense to start as a CLI flag for pepr build, instead of switching the default. Starting point would be a new option like https://github.com/defenseunicorns/pepr/blob/8ad66b86e40f449acaea9c14792b12e8f26749cd/src/cli/build.ts#L55
Likely -z, --zarf [<manifest|helm chart>] or similar.


Update the API for I/O safety

Add new terminal_size_of fuctions which use the AsFd and AsHandle traits to accept file descriptors (on Unix) and handles (on Windows).
Deprecate the existing terminal_size_using_* functions which take raw file descriptors or handles without being unsafe.
And update the get_size example to use the new terminal_size_of functions. The example can now be much simpler because the main API is now the same between Windows and Unix in common cases.

Thanks!  I'm sure the code is fine, but I want to take a moment to make sure I understand it.   Target for merge and release is sometime next weekend.

Thanks again for this PR (and for all your work on pushing the I/O safety stuff)


Converting Scorcher into Dawnbringer

Reworks existing unit into a more distinct anti-surface/anti-crowd role by turning its attack into a set of wandering beams that deal damage, but tend not to dwell on one target for long.
Currently fires 1-5 beams (increasing by 1 every 1.5s), dealing 5 damage at their endpoints every 0.2s.
Known bugs:

No warp-in model. This is due to the addition of the GimbalZ actor, which allows it to track targets and aim up and down while moving; without that actor, beam movement tends to look a little odd. Haven't found a workaround yet.

Fixed issues, and also did some actor changes that should make the turn-and-target-in-place visual effect a little less janky.

Bug: Multiple units don't stack on same target

Bug: Multiple units don't stack on same target

Fixed, thanks for pointing that out


resubscribe logic looks ignoring sharded Pub/Sub (shardChannels Set of PubSubEndpoint)

Bug Report
Thank you for adding support for sharded PubSub in the 6.4.0.
Related to that implementation, I would like to confirm about resubscribe (subscription recovery after reconnection) behavior.
It looks like resubscribe handling only non-sharded PubSub subscriptions.
Current Behavior
Sharded PubSub subscriptions looks not recovered after disconnection + re-connection, despite Lettuce automatically recovers non-sharded PubSub subscriptions (the recovery behavior is very helpful for applications, and also difficult / error-prone to implement by ourselves).
In the https://github.com/redis/lettuce/commit/1457486165edb8e7184c0214dae9973c7f6ff567#diff-73f0c5ac0f7d6d94f8d47e1a75564aa344cbbbd4c91ce786711b5c3673044956, PubSubEndpoint.shardChannels (private) have been added but it looks that variable is private inside PubSubEndpoint and no any code of PubSubEndpoint reading that variable's data. Also implementations of StatefulRedisPubSubConnectionImpl.resubscribe and StatefulRedisClusterPubSubConnectionImpl.resubscribe does not accessing shardChannels.
Input Code
Just make sharded Pub/Sub connection, and wait until some disconnection + re-connection occurs on that environment...

Input Code
StatefulRedisPubSubConnection<String, String> connection = client.connectPubSub()
connection.addListener(new RedisPubSubAdapter<String, String>() { ... })

RedisPubSubCommands<String, String> sync = connection.sync();
sync.ssubscribe("channel"); // Sharded subscription

Expected behavior/code
As same as non-sharded Pub/Sub, automatically recover sharded Pub/Sub subscriptions when possible to connect to that shard/node.
( May be a bit different logic from non-sharded Pub/Sub needed. Because the SSUBSCRIBE requires living connection to the particular shard's node(s) )
Environment

Lettuce version(s): 6.4.0
Redis version: 7.1 (I belive this issue happens even with Redis 7.0)

Possible Solution
(Not tested) Watch the connection's state somehow and manually call ssubscribe again by application's logic.
Additional context
Because currently I don't have full access to a target environment / Redis cluster, this issue is based on best guess by reading code. I hope this code reading makes some sense...

(Just related topic) Also Javadoc of RedisClusterClient#connectPubSub and StatefulRedisClusterPubSubConnection are bit conflicting each other. I assume the StatefulRedisClusterPubSubConnection one is correct:
RedisClusterClient#connectPubSub:

 *
Pub/sub commands are sent to the node with the least number of clients

StatefulRedisClusterPubSubConnection:

The connection provides transparent command routing based on the first command key.

↑ By reading code, this one (routing based on first command key) is the truth and that is why ssubscribe (but also subscribe, despite it can be sent to any shard/node) implementation always sends command to the key slot's shard/node

Hey @saiya , thanks for the report, I think you are correct and sharded subscriptions are not resubscribed right now.
I will put this in our backlog so we can address it as soon as time permits.

By reading code, this one (routing based on first command key) is the truth and that is why ssubscribe (but also subscribe, despite it can be sent to any shard/node) implementation always sends command to the key slot's shard/node

Both, to the best of my own understanding:

channel names are keys and the command will be send to a node handling the slot that handles this key
in case more than one shard is handling this slot (replica nodes) we subscribe to the one with the least number of clients

Does that make sense?

Thank you for the agile response! I will await for the fix.

Both, to the best of my own understanding:

channel names are keys and the command will be send to a node handling the slot that handles this key
in case more than one shard is handling this slot (replica nodes) we subscribe to the one with the least number of clients
Does that make sense?

I see; yes indeed that make sense.
Now I understand the RedisClusterClient#connectPubSub document's intention; Another list item saying
Keyless commands are send to the default connection
, therefore that list item talking about not-keyless command implictly.


Fix call signature on Gate to show that it takes Qids.

This makes the API documentation for our gate constants better.  Not sure why it trips the type notation.

@tanujkhattar PTAL
I think this is the minimal change. It leaves the signatures the same, and to get around the CXPow gate it adds an explicit override for the __call__, which is not horrible.  There is also a change necessary in GateOperation repr for an assert, but I think that assert is way too defensive.

So, what we have now is also a breaking change because every user defined class which was dependant on Gate.__call__ to forward the **kwargs to it's on method (like CXPowGate) will now break. We would ideally need to go through a deprecation cycle to make this change to avoid breaking user code directly.

So, what we have now is also a breaking change because every user defined class which was dependant on Gate.__call__ to forward the **kwargs to it's on method (like CXPowGate) will now break. We would ideally need to go through a deprecation cycle to make this change to avoid breaking user code directly.

Hm, not sure I agree about that.  This doesn't break any public interface, and if someone chose to follow the pattern of CXPowGate, they would be doing so in a manner that isn't really explicitly supported by any documented interface on Cirq's end.

I'm not a huge fan of this change solely in the name of getting better type hints (which has no functional impact by itself). There's also no super clear guidance on type hinting for *args and **kwargs. If it's just for docs, what if we just added type hints and then immediately washed them out with ignore comments ? That would get us the working docs (hopefully) and keep existing behavior.

I'm not a huge fan of this change solely in the name of getting better type hints (which has no functional impact by itself). There's also no super clear guidance on type hinting for *args and **kwargs. If it's just for docs, what if we just added type hints and then immediately washed them out with ignore comments ? That would get us the working docs (hopefully) and keep existing behavior.

I'm not sure I follow.  Why would we ignore the type hints?

So that we could fix the API docs issue without having to introduce any code changes until we can come up with a fix everyone was on board with. Maybe I'm missing something here, is this a docs motivated PR or do we want the code fix itself in this PR too ?

So that we could fix the API docs issue without having to introduce any code changes until we can come up with a fix everyone was on board with. Maybe I'm missing something here, is this a docs motivated PR or do we want the code fix itself in this PR too ?

The primary goal is to make sure all the gate constants have a reasonable doc string which shows they can be called on qubits.  The type gets dropped by our beloved tensorflow doc overlords :)
This doesn't end up changing any interface and really just changes where passing in kwargs fails for on and __call__.  Prior to this it will fail with a python error about on not supporting this if you call it on on or __call__.  After this change it will fail by yelling about on when you call on and __call__ when you call _call__.  So for tracing problems, it is actually slightly better.  For overloading these methods if you want to accept kwargs, you will have to overload both now (this is what is done in the CX case).  This reads somewhat more self contained, but is slightly worse from a code reuse perspective.
Really there was a bug as they should have been consistent with each other but fixing that is a breaking change.

@MichaelBroughton let me know if you are OK with this, if not, also great and I will close this :)


Using WH3 MM Causes In-Ability To Join Multiplayer Battles/Campaigns

When using mods on the WH3 MM It shows that our modlist are incompatible even when importing the modlist and everything, we've deleted mods redownloaded, verified files, etc
However it worked when we used the default WH3 Launcher with the EXACT same modlist and load order

Does one of you have skip intro movies enabled? It shows up as a mod in the .txt file.

I don't think so

I've been trying to get multiplayer to work with this mod manager since 4.1 released, and I've found a workaround for the modlist showing as incompatible. In the WH3 root folder there's "used_mods.txt" that the mod manager appears to edit, if the contents of that file don't match exactly between all players in multiplayer it will show as incompatible. By making sure that the mods appear in the text file in the same order for all players, it should work. I have tested this both with skip intro movies enabled and disabled.


Proposal: Move vdrtools fork under aries-vcx repo

I'd like to suggest moving the fork of vdr-tools into this repo. We currently using fork of vdrtools, eg:
https://github.com/hyperledger/aries-vcx/blob/0.50.0/aries_vcx/Cargo.toml#L40
Given good developments toward credex libs migration and commitment to drop usage of libindy/vdrtools in 2023, in combo with the fact there's no updates for the library from Evernym https://gitlab.com/evernym/verity/vdr-tools I think we can continue taking lead within our fork, but move the development here, rather than having it externalized and less transparent on some gitlab repo.
As underlying code for holder, prover, verifier works with credex libraries, large portions of the vdr-tools codebase can technically be deleted today (with some updates to libvcx codebase to start using credex/indy-vdr codepaths by default). By the end of the journey, the only thing which should be left out of vdrtools is wallet implementation, which might be then superseded (or might be used interchangeably) by (or with) Aries Askar.
Additionally having git dependency would become a problem once we attempt to publish aries-vcx as crate (this was touched on at the last aries-vcx call)
@mirgee @gmulhearn what do you think?

I am confident M1 user won't notice ;-) But on slower machines, I guess depends on IDE, both performance and possible solutions. in InteliJ you can exclude directory from being indexed.


Use waves as math functions in A2S

Add a construct wave [phase], which returns the interpolated sample value at phase, where phase is 0 for the start of the wave, 1.0 for the start of the second period of the wave etc. (1.0 would be the wrapping point for a single cycle wave, like the built-in geometric waveforms.)
Wrapping or clamping is applied depending on the loop settings for the wave, just like when it's played by 'wtosc'.

Should probably implement #51 instead/first, as that pretty much covers this (if a bit awkwardly in some cases), seems much more useful in the context of sound and music, and takes care of the sample rate/miplevel problem.


pass logger in but default to console.

This adds a new option to overwrite the logger away from console.
I passed the loggerinterface around everywhere and fixed tests.

Merged with changes in https://github.com/statsig-io/node-js-server-sdk/commit/c9363732065da22ed0be15f0fa394307b74b34f8

Thanks.


Added page description for better lighthouse score

This take it from 80 => 90, and paired with #77 SEO score will be 100.

example of how it will read: <meta name="description" content="fonts designed for writing code, Anonymous Pro">

Definitely down to have a good meta tag here!
Updated the text a smidge.


[BUG] Incompatibility with Perfect Vision's sight limit

Module Version: v1.4.0
Describe the bug
This module prevents the sight limit of Perfect Vision from functioning as intended
To Reproduce
Steps to reproduce the behavior:

Install both Advanced Templates and Perfect Vision
Place down a template
Within the template configuration, Click on Configure Vision Limitation and set to desired distance (e.g. 5ft)
Place token within the template and select them to see what they can see

Expected behavior
The token's vision should be impacted by the Configure Vision Limitation setting from Perfect Vision.
Screenshots
Without module enabled:

With module enabled:

Browser:
Foundry Client/Firefox
Foundry Version:
10.291
Game System:
0.82.5
Additional context
N/A

Interesting. I'll install that mod and see if I can figure out what's going on

Sorry for the late response, but as Perfect Vision is no longer supported for Foundry, and this mod has been updated for v11, there basically is no longer an incompatibility for a mod that doesn't exist for the current version.


Create SigmaReplacements-Textures.netkan

Does this conflict with TextureReplacer, TextureReplacerReplaced, or TexturesUnlimited?

no it shouldn't conflict with those


Endorsed mechanism - documentation enhancement

I have downloaded 1.3. The documentation reads:
https://metro.dev.java.net/nonav/1.3/docs/ReleaseNotes.html#Running_on_top_of_JDK_6
6. Running on top of JDK 6
JDK 6u4 includes JAX-WS 2.1 and JAXB 2.1 API and RI. Metro 1.3 FCS
includes newer and enhanced RI of JAX-WS/JAXB 2.1. In order to run
Metro 1.3 FCS you would need to put the jars in the classpath(endorsed
mechanism is not required anymore).
At the other place of the documentation:
https://metro.dev.java.net/nonav/1.3/docs/jaxws-tools.html
Running on JDK6

Copy METRO_HOME/lib/jaxws-api.jar to JDK6_HOME/jre/lib/endorsed directory
Copy METRO_HOME/lib/jaxb-api.jar to JDK6_HOME/jre/lib/endorsed directory
For details see Endorsed Directory Mechanism. Above, METRO_HOME points
to the root directory of the extracted Metro bundle. JDK6_HOME points
to JDK 6 installation directory.

If I look into the "lib" directory there are no such as JARs. I can
only find webservices-*.jar files there.
If I try to run a tool I am getting: You are running on JDK6 which
comes with JAX-WS 2.0 API, but this tool requires JAX-WS 2.1 API. Use
the endorsed standards override mechanism
(http://java.sun.com/javase/6/docs/technotes/guides/standards/), or
use -Xendorsed option.
It should be clearified. I also do not understand why endorsed mechanism has to
be used with tools and not with runtime.
Environment
Operating System: All
Platform: All
Affected Versions
[1.3]
Source: https://github.com/javaee/metro-wsit/issues/1004
Author: glassfishrobot

endorsed is no longer needed


update glob and browserify to fix minimatch security issues

Reduce the scope of https://github.com/clutchski/coffeelint/pull/573
Resolves security issues with old nested minimatch dependencies. Vows still hasn't updated to a newer minimatch but it is a development dependency so that shouldn't cause any issues.

@AsaAyers Thanks for the merge! If any issues come up because of this feel free to ping me so I can fix them.


Optimization strategy for continued training

Hi,
First of all, thank you for sharing your work!
Do you have any recommendations on optimizer and learning rate scheduling configurations for continuing the causal language model to another language? I'm planning on continuing the language modeling pretraining on Portuguese data.
If there is already documentation covering this, please send it to me, and I will look into it.
Best,
Marcos

We haven't tried continuing the pre-training yet, so I'm not so sure which hyperparameter setting would work the best. However, we are planning to do it after we reach 1T tokens, and I'll update you about the configurations we use.

I think math data can be generate in every single calculation between -1000 ~ 1000, and random calculations in larger scale.
No clue how that model can handle math after that...

Thanks, @young-geng !

An update: we've decided to continue training the model on a mixture of natural language and code. We are using the same learning rate schedule as the previous training run.

Perfect @young-geng , thanks!!

@young-geng do you plan to release any documentation regarding the training process?

@ahmedrshdy Our pretraining configuration is basically this:
https://github.com/young-geng/EasyLM/blob/main/examples/pretrain_llama_7b.sh


Ema/fix script naming

Reviewer: @shakyShane
Asana: NA
Description
Just a fix in the script naming I forgot to update.
Steps to test

@GioSensation did this include extra things by mistake?

🤦♂ Fixed.


#10 invertible matrix returns null

This PR is related to #10
After reading the code, I feel that the original logic was to return 'null'. So if followed this pattern.
I've also faced an infinite loop while running npm test so i fixed that also.
@paulmelnikow, please share your feedbacks

any update ?

Sorry I missed this! Thanks for the nudge 😄


Conflict with last-applied-configuration annotation when merging

Describe the solution you'd like
It should merge successfully
What is the added value?
Merging secrets will work in more cases
Give us examples of the outcome
kind: ExternalSecret
metadata:
  name: secret
  labels:
    app.kubernetes.io/name: secret
    app.kubernetes.io/part-of: app
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: gcp
  target:
    name: secret
    creationPolicy: Merge
  data:
  - secretKey: webhook.github.secret
    remoteRef:
      key: webhook-secret

Message:                unable to patch secretsecret: Apply failed with 1 conflict: conflict with "controller" using v1: .metadata.annotations.kubectl.kubernetes.io/last-applied-configuration
    Reason:                 SecretSyncedError

Observations (Constraints, Context, etc):
The controller is updating other keys within the secret. I am trying to populate this one value from GCP. If I delete the conflicting annotation it works until the controller updates the secret again.
apiVersion: v1
kind: Secret
metadata:
  name: secret
  namespace: app
  selfLink: /api/v1/namespaces/app/secrets/secret
  uid: dd
  resourceVersion: '155912867'
  creationTimestamp: '2021-10-17T04:24:36Z'
  labels:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: secret
    app.kubernetes.io/part-of: app
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"app","app.kubernetes.io/name":"secret","app.kubernetes.io/part-of":"app"},"name":"secret","namespace":"app"},"type":"Opaque"}
    reconcile.external-secrets.io/data-hash:<PHONE_NUMBER>2222333333
  managedFields:
    - manager: external-secrets
      operation: Apply
      apiVersion: v1
      time: '2022-01-12T18:54:31Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          f:webhook.github.secret: {}
        f:immutable: {}
        f:metadata:
          f:annotations:
            f:reconcile.external-secrets.io/data-hash: {}
          f:labels:
            f:app.kubernetes.io/instance: {}
            f:app.kubernetes.io/name: {}
            f:app.kubernetes.io/part-of: {}
    - manager: controller
      operation: Update
      apiVersion: v1
      time: '2022-01-12T18:55:29Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            f:kubectl.kubernetes.io/last-applied-configuration: {}
immutable: false
data:
  accounts.tokens: xxx
  server.secretkey: xxx
  webhook.github.secret: xxx
type: Opaque

hey @lancehudson, i can reproduce your issue. It seems like there is an issue when mixing operation=Update and operation=Apply; It's best described here: https://github.com/kubernetes/kubernetes/issues/80916#issuecomment-525049458 - it looks like we can't do server-side apply with client-side applied resources.
I think we should force ownership of the fields we care about (as described in the docs). I prepared a patch to fix this issue.


ctx.body assignment of stream not working

Koa: 2.13.4
Node: 14.18.0
ctx.attachment('logo.png');
ctx.body = fs.createReadStream('static/logo.png');

It returned text of structure stream instead of binary image content:
StatusCode        : 200                                                                                StatusDescription : OK
Content           : {137, 80, 78, 71...}
RawContent        : HTTP/1.1 200 OK
                    Transfer-Encoding: chunked
                    Content-Disposition: attachment; filename="logo.png"
                    Connection: keep-alive
                    Keep-Alive: timeout=5
                    Content-Type: image/p...
Headers           : {[Transfer-Encoding, chunked], [Content-Disposition, attachment; filename="logo.png"], [Connection, keep-alive], [Keep-Alive, timeout=5]...}
RawContentLength  : 1544

could you provide the minimum reproductive repository? Because regarding headers you've attached to this issue everything is working fine and should signal the client to prompt for download

Closing this as stale as a minimal reproducible example was not provided per vbetsuns request! Reopen or create a new ticket if you still believe there's an issue to discuss!


fix: lowercase fine-tuning-6b deployment name due to CDK resource naming constraints

Describe your changes

lowercase fine-tuning-6b deployment name due to CDK resource naming constraints

Checklist before requesting a review

[x] I updated CHANGELOG.MD with a description of my changes
[x] If the change was to a module, I ran the code validation script (scripts/validate.sh)
[x] If the change was to a module, I have added thorough tests
[x] If the change was to a module, I have added/updated the module's README.md
[x] If a module was added, I added a reference to the module to the repository's README.md
[x] I verified that my code deploys successfully using seedfarmer apply

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
Commit ID: aadae520b3483cb810867588a7893ff7664f53e7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AIOpsModuleTestsInfrastruct-doMLXEYsnmxr
Commit ID: a58d1534331497af53088218b234beadf123049b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Add navigation

Currently, there's no navigation in the app, even though users can access https://nwc.getalby.com/about through a link.
Let's add "Sessions" and "About" items into the nav bar. How it should be behave and what to display with logged out users can be seen in the Figma file.

We don't have a user address/avatar currently. but we can add the navigation.
@im-adithya do you want to check this out.


All validation except one instance is ignored if validation elements have no Id attribute

I've been trying out the example in the flutter_dynamic_forms_components and stumbled upon an issue relating to validation.
The example forms in the assets folder have validation elements that do not specify id property attributes.  The result of not having Ids results in the code here producing a Map with only one Validation element, whichever is the last to be yielded.  This is because the key values were null values and you can't have duplicate keys in a Map.
If I provide unique id values in the xml data then all the validations are included correctly.  So I guess I'd suggest to enforce Validation elements have an Id or change the FormManager constructor to accept a list of Validations rather than a Map.  What do you think @OndrejKunc ?
Great library by the way.

Hi @markphillips100 ,
Thanks for the feedback!
I am aware of this issue. Besides validation, it also causes problems when collecting data from all the inputs like a checkbox - when you forget to add an id, it doesn't really make sense to send it back to the server, because the server wouldn't be able to identify the source component.
Unfortunately, I don't know about any easy way how to enforce having an id, because we would need to throw some exception during parsing, and that's probably not wanted. Another solution might be to generate some random id when it is missing, but I think that maybe confusing, because it would be different every time. Or maybe instead of random id, it can be the unique path to the element in a tree, something like XPath, but it would not be so easy to implement.
However in case of validations, most of the time we really don't need the id, we just need to check if all validations are valid. Changing it to a list sounds like a good idea! Right now I am rewriting the way how the FormManager is built as part of #74 so I think I might also include this change there.

I added the change from map to the list via PR #78  and published it to pub.dev
I hope it solves your problem.

Awesome @OndrejKunc .  I've upgraded to 0.13.0 and all is good.  Thank you.


Several OAuth Envrionment Variables Incorrect

Hi!
After reviewing the docs:
https://docs.linkwarden.app/self-hosting/sso-oauth#github
I've found that several of these environment variables are incorrect.
For instance, the GitHub provider on your docs state:
GITHUB_CLIENT_ID
GITHUB_CLIENT_SECRET

But after reviewing the next auth docs (https://next-auth.js.org/providers/github) here's what they say:
GITHUB_ID
GITHUB_SECRET

Thanks for pointing this out, just fixed it.


Add Yugabyte DB

I see TiDB and Cockroach listed as NewSQL, and I think that yugabyte has a lot of promise
https://www.yugabyte.com/
https://github.com/YugaByte/yugabyte-db

The high-performance distributed SQL database for global, internet-scale apps.

@camerondavison feel free to add a PR, closing issue.


Why can’t I download lossless audio but the telegram bot can?

The bot is too slow for my use case (downloading >2000 songs), but gamdl fails to download ALAC .m4a files or any other lossless files when I use “ask” instead of “alac”. I already have mp4decrypt, so why doesn’t it work?
I also tried the exact same url with the Glomatico bot and it worked. It was slow, but it still got me the lossless file.

The API returns manifests that don't have Widevine keys for some songs and when the Widevine keys are present the license serve doesn't return the license server.

So is there a way for me to download lossless using gamdl or do I have to use the bot bit by bit?


Get no results with mv_getgroupmembers

Hi Allan,
I am trying to get a list of all groupmembers (SAS Viya 3.4). I have followed the instructions on https://sasjs.io/buildviya/. I have ran the SAS code on the following url: https://raw.githubusercontent.com/sasjs/core/main/all.sas and I then ran the following macro- %mv_getgroupmembers(All Users) (https://core.sasjs.io/mv__getgroupmembers_8sas.html).
But I don't get result:

Hi @Sergie89,
I just tried this on a Viya 4 instance and the macro appears to be working as designed (I had thought that maybe the space was an issue, but it is not).
I did not have an "All Users" group, however it did work for other groups.  Did you try the macro with any other groups?  Is it possible you don't have an "All Users" group, or that you don't have permission to access it?
If you are looking to extract all the users, rather than the members of a particular group, you are probably better off using this macro:  https://core.sasjs.io/mv__getusers_8sas.html
/Allan

Hi @allanbowe ,
It works thanks. Is it possible to get a list of all groups & group members with one single SAS macro ?

Thanks for confirming!
To your question - yes it's possible, just use this macro to get the lists of groups:  https://core.sasjs.io/mv__getgroups_8sas.html
Then loop through the result with mv_getgroupmembers to get the members.
Feel free to make a PR should you create such a macro!


CXXCBC-173: complete streaming lexer even if pointer didn't match

Looks good except for the failing test on 6.6.5


feat(docs): Remove landing page

Description
Removes the previous landing page for the docs and send them to the first page instead.
Problem*
Resolves https://github.com/noir-lang/noir/issues/4982
Summary*
I added this plugin for redirecting the landing page to the root of the docs. I am open to other suggestions, but we should preserve the existing url structure so links don't break everywhere.
Check one:

[ ] No documentation needed.
[x] Documentation included in this PR.
[ ] [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

[ ] I have tested the changes locally.
[x] I have formatted the changes with Prettier and/or cargo fmt on default settings.

That would break existing links

Also, if you check out the deploy, when you first click it it correctly goes to /docs, but if you click the logo in the top left it goes to / and says "page not found". I'm open to suggestions

@critesjosh, is this PR being developed further or should I close it?

After chatting with @Savio-Sou, we are going to keep the landing page as this is the only website for noir and landing directly on a docs homepage is not ideal. We will likely want to update the style of the landing page with new branding, but that will be another PR.


Given two people find shortest chain of relationships

Six degrees of separation is the theory that everyone on Earth is six or fewer steps away, by way of introduction, from any other person in the world, so that a chain of “a friend of a friend" statements can be made to connect any two people in a maximum of six steps (or five intermediaries).
On the social network: given two people of the network, your task is to retrieve the shortest chain that relates them.

Finished. Please review.

Finished. Please review.

Also added a hidden option (0) to automatically add people55 and friends55 at execution time.
Love that idea ❤

Also added a hidden option (0) to automatically add people55 and friends55 at execution time.
Love that idea ❤

Also added a hidden option (0) to automatically add people55 and friends55 at execution time.

Love that idea ❤

Finished. Please review.

I have reviewed the code and implemented mine too, tomorrow we will comment what to add to master and interesting options.

TODO: Delete Relation class as it is obsolete now?

I have deleted in my branch, it has no sense to use it, but general concept must be changed (I have done that on my branch)

Also added a hidden option (0) to automatically add people55 and friends55 at execution time.

Love that idea ❤

Finished. Please review.

I have reviewed the code and implemented mine too, tomorrow we will comment what to add to master and interesting options.

TODO: Delete Relation class as it is obsolete now?

I have deleted in my branch, it has no sense to use it, but general concept must be changed (I have done that on my branch)

Merged

Merged


Lizard tail sprites clip over magboot sprites

Description
Reproduction

Spawn as a lizard
Have a tail
Equip magboots

Screenshots

what if make tails a body parts?

what if make tails a body parts?

This will most likely not fix that.

dupe of #15132


Fix PDOConnector GeneratedID return type

Alternate Version of #8872
Differences:

use (int) instead of intval()
removed blank line from unit test

Description:
After inserting a DataObject into a database the ID of that object should be an int.
PDOConnector uses PDO::lastInsertId to retrieve the ID. PDO::lastInsertId returns a string. This fix converts the return value to int to make it compliant with the DBConnector::getGeneratedID docs.
To reproduce this issue:
Create a new DataObject
Write it to the DB.
Check the type of the ID Field.

Looks good, php-cs failure is unrelated to these changes


cmd/juno: add CLI tools for db

Description
This PR adds a new CLI tool for database-related operations (i.e. juno db [commands]). Under the db category, there are 2 subcommand:

juno db info - display the blockchain information stored in the database

{
  "network": "sepolia",
  "chain_height": 71958,
  "latest_block_hash": "0x69fa4f792a98cab20d5fe5040401a8377ae75234ee2446590e1b68d3abfa9e2",
  "latest_state_root": "0x7c5dd3a676f5b6f603227b1f1b77f8d95a96b05abc93c09f0715d27c7169f1c",
  "l1_height": 66367,
  "l1_block_hash": "0x335b4136b1da8317a1a72450ddb2df813ae828cb2ee4347d4c21d2d8adee0dc",
  "l1_state_root": "0x4ff66961f76f33cedc65124cbb3021e370cd966434c13245f1d75015e136ba8"
}

juno db size - display the Inspect the database to obtain storage information

Calculating size of StateTrie, remaining buckets: 23
Calculating size of Peer, remaining buckets: 22
Calculating size of ContractClassHash, remaining buckets: 21
Calculating size of ContractStorage, remaining buckets: 20
Calculating size of Class, remaining buckets: 19
Calculating size of ContractNonce, remaining buckets: 18
Calculating size of ChainHeight, remaining buckets: 17
Calculating size of BlockHeaderNumbersByHash, remaining buckets: 16
Calculating size of BlockHeadersByNumber, remaining buckets: 15
Calculating size of TransactionBlockNumbersAndIndicesByHash, remaining buckets: 14
Calculating size of TransactionsByBlockNumberAndIndex, remaining buckets: 13
Calculating size of ReceiptsByBlockNumberAndIndex, remaining buckets: 12
Calculating size of StateUpdatesByBlockNumber, remaining buckets: 11
Calculating size of ClassesTrie, remaining buckets: 10
Calculating size of ContractStorageHistory, remaining buckets: 9
Calculating size of ContractNonceHistory, remaining buckets: 8
Calculating size of ContractClassHashHistory, remaining buckets: 7
Calculating size of ContractDeploymentHeight, remaining buckets: 6
Calculating size of L1Height, remaining buckets: 5
Calculating size of SchemaVersion, remaining buckets: 4
Calculating size of Pending, remaining buckets: 3
Calculating size of BlockCommitments, remaining buckets: 2
Calculating size of Temporary, remaining buckets: 1
Calculating size of SchemaIntermediateState, remaining buckets: 0
+-----------------------------------------+------------+----------+
|                 BUCKET                  |    SIZE    |  COUNT   |
+-----------------------------------------+------------+----------+
| StateTrie                               | 9.44 MiB   |   143032 |
| Peer                                    | 0.00 B     |        0 |
| ContractClassHash                       | 4.43 MiB   |    71516 |
| ContractStorage                         | 662.14 MiB |  5970298 |
| Class                                   | 5.80 GiB   |    11139 |
| ContractNonce                           | 4.43 MiB   |    71516 |
| ChainHeight                             | 9.00 B     |        1 |
| BlockHeaderNumbersByHash                | 2.88 MiB   |    73536 |
| BlockHeadersByNumber                    | 115.86 MiB |    73536 |
| TransactionBlockNumbersAndIndicesByHash | 91.42 MiB  |  1956435 |
| TransactionsByBlockNumberAndIndex       | 4.03 GiB   |  1956435 |
| ReceiptsByBlockNumberAndIndex           | 4.82 GiB   |  1956435 |
| StateUpdatesByBlockNumber               | 688.26 MiB |    73536 |
| ClassesTrie                             | 1.14 MiB   |    17370 |
| ContractStorageHistory                  | 848.86 MiB |  8477125 |
| ContractNonceHistory                    | 43.72 MiB  |   628061 |
| ContractClassHashHistory                | 57.53 KiB  |      807 |
| ContractDeploymentHeight                | 2.80 MiB   |    71516 |
| L1Height                                | 0.00 B     |        0 |
| SchemaVersion                           | 9.00 B     |        1 |
| Pending                                 | 1.56 MiB   |        1 |
| BlockCommitments                        | 8.56 MiB   |    73536 |
| Temporary                               | 0.00 B     |        0 |
| SchemaIntermediateState                 | 2.00 B     |        1 |
+-----------------------------------------+------------+----------+
|                  TOTAL                  | 17.08 GIB  | 21625833 |
+-----------------------------------------+------------+----------+
+-----------------+----------+----------+
|      STATE      |   SIZE   |  COUNT   |
+-----------------+----------+----------+
| Without history | 6.46 GiB |  6267501 |
| With history    | 7.33 GiB | 15373494 |
+-----------------+----------+----------+

Can we also show to which network db corresponds to? it would be great if there is some snapshot stored without identification

Can we also show to which network db corresponds to? it would be great if there is some snapshot stored without identification

I think that's a good addition. From my understanding, the network ID isn't stored directly in the db. Let me figure out how to pull that information.

Can we also show to which network db corresponds to? it would be great if there is some snapshot stored without identification

I think that's a good addition. From my understanding, the network ID isn't stored directly in the db. Let me figure out how to pull that information.

That's right you can modify existing approach for your case:
// Verify that cfg.Network is compatible with the database.
	head, err := chain.Head()
	if err != nil && !errors.Is(err, db.ErrKeyNotFound) {
		return nil, fmt.Errorf("get head block from database: %v", err)
	}
	if head != nil {
		// We assume that there is at least one transaction in the block or that it is a pre-0.7 block.
		if _, err = core.VerifyBlockHash(head, &cfg.Network); err != nil {
			return nil, errors.New("unable to verify latest block hash; are the database and --network option compatible?")
		}
	}

You should iterate over networks to figure out (core.VerifyBlockHash(...) == nil) to which one it corresponds

You should iterate over networks to figure out (_, err := core.VerifyBlockHash(...); err == nil) to which one it corresponds
Added the network field.

TODO for myself: do not find the target directory using working directory, binary will not be able to run anywhere. Perhaps use embed directive.

Calculating DB size has some differences between this implementation and https://github.com/NethermindEth/juno/pull/1773:

Formats the data size into metric prefixes (i.e. MiB, GiB, etc)
Adds count of each KV pair
Outputs result in table format
Put this subcommand under the subcommand of db ./juno db size instead of ./juno db-size

Tests fail because https://github.com/NethermindEth/juno/pull/1978 has to be merged.


Do we need to keep the trademark on SciGraph to keep this community named?

Please let me know ASAP if you would like to keep this trademark.
Springer has a thing called SciGraph, but we have the trademark for the name.
If we want to keep it, I need to have a specimen (a screenshot) for how our version of SciGraph is being used in something that could be considered "trade". I think that it would work if we have any commercial or non-commercial groups using our version of SciGraph.
If I let this go, I would anticipate that Springer/Nature will make us change our name.

It is used in SPARC serving SCKAN (publicly documented APIs using SciGraph) so that would probably be considered trade.


Add h10n.me to the showcase

Hi @zerok,
thank you for adding your website to the showcase of Hugo. I've merged your addition as 64ed64d93f77d2defa49d24d79aa9e1ddb28ab9b (master) and fdfac32057fac69fbb6e6b4998cb9ba2aa7998cc (v0.15.docs).
The showcase should be updated within the next minutes.
Cheers,
Digitalcraftsman

@digitalcraftsman Thank you very much :smiley: