[dask] Use inplace prediction by default.

Use inplace predict for dask skl interface.
Fix https://github.com/dmlc/xgboost/issues/6553 by using best_iteration.  The issue was caused by ignoring ntree_limit, but this argument is confusing in the context of forest and multi-class, so I decided to avoid it for any new code.

To-do:

[ ] Add tests for early stopping.

Does it mean that pred_contrib is not available on Dask?

Does it mean that pred_contrib is not available on Dask?

@trivialfis FYI ntree_limit is a critical aspect of xgboost and using early stopping.  The tests in xgboost make it clear how ntree_limit is connected the value of best_iterations.

@trivialfis FYI ntree_limit is a critical aspect of xgboost and using early stopping.  The tests in xgboost make it clear how ntree_limit is connected the value of best_iterations.

Does it mean that pred_contrib is not available on Dask?
See #6478

FYI the scikit-learn API for dask does not support pred_contribs=True, just output_margin and base_margin.

Does it mean that pred_contrib is not available on Dask?
See #6478

FYI the scikit-learn API for dask does not support pred_contribs=True, just output_margin and base_margin.

In my opinion we can just forget the existence the ntree_limit.  Could you please take a look into model slicing?  Let us know if there's any use case for ntree_limit that can not be replaced by model slicing.

In my opinion we can just forget the existence the ntree_limit.  Could you please take a look into model slicing?  Let us know if there's any use case for ntree_limit that can not be replaced by model slicing.

In my opinion we can just forget the existence the ntree_limit. Could you please take a look into model slicing? Let us know if there's any use case for ntree_limit that can not be replaced by model slicing.

This?  https://xgboost.readthedocs.io/en/latest/python/model.html
This is even further far away from any sklearn API and has all the same complications w.r.t. tree counts as ntree_limit.

In my opinion we can just forget the existence the ntree_limit. Could you please take a look into model slicing? Let us know if there's any use case for ntree_limit that can not be replaced by model slicing.

This?  https://xgboost.readthedocs.io/en/latest/python/model.html
This is even further far away from any sklearn API and has all the same complications w.r.t. tree counts as ntree_limit.

https://github.com/dmlc/xgboost/issues/6568 FYI pred_contribs doesn't work with non-sklearn dask API as well.

https://github.com/dmlc/xgboost/issues/6568 FYI pred_contribs doesn't work with non-sklearn dask API as well.

@trivialfis if the proposal is to (by default) always have predict use the best_iteration result, so user gets the optimal early stopping result by default, that is a good proposal.  If normal xgboost also did that, this would be a good change because the best iteration result is the normal thing wanted when doing early stopping, not the total number of trees one ends up with.  If in that case one had to use model slicing in order to get some arbitrary other non-best tree prediction, then that is an ok proposal.  But it has to be made uniform between dask and non-dask.
Also, then one should avoid needing to have to pass ntree_limit=0 for GLM, which is a pretty silly requirement.

@trivialfis if the proposal is to (by default) always have predict use the best_iteration result, so user gets the optimal early stopping result by default, that is a good proposal.  If normal xgboost also did that, this would be a good change because the best iteration result is the normal thing wanted when doing early stopping, not the total number of trees one ends up with.  If in that case one had to use model slicing in order to get some arbitrary other non-best tree prediction, then that is an ok proposal.  But it has to be made uniform between dask and non-dask.
Also, then one should avoid needing to have to pass ntree_limit=0 for GLM, which is a pretty silly requirement.

This is even further far away from any sklearn API and has all the same complications w.r.t. tree counts as ntree_limit.

Really?  To me ntree_limit to me is much more complicated.  Do you calculate this value by hand when using cls with forest?

the proposal is to (by default) always have predict use the best_iteration result, so user gets the optimal early stopping result by default, that is a good proposal.

So far that's the default for single node scikit interface.  Just making the alignment here.

But it has to be made uniform between dask and non-dask.

Of course.  We are making progress on the unification.  The 2 interfaces are now much more closer than a few months before.

Also, then one should avoid needing to have to pass ntree_limit=0 for GLM, which is a pretty silly requirement.

Agree.

This is even further far away from any sklearn API and has all the same complications w.r.t. tree counts as ntree_limit.

Really?  To me ntree_limit to me is much more complicated.  Do you calculate this value by hand when using cls with forest?

the proposal is to (by default) always have predict use the best_iteration result, so user gets the optimal early stopping result by default, that is a good proposal.

So far that's the default for single node scikit interface.  Just making the alignment here.

But it has to be made uniform between dask and non-dask.

Of course.  We are making progress on the unification.  The 2 interfaces are now much more closer than a few months before.

Also, then one should avoid needing to have to pass ntree_limit=0 for GLM, which is a pretty silly requirement.

Agree.

This is even further far away from any sklearn API and has all the same complications w.r.t. tree counts as ntree_limit.

Really? To me ntree_limit to me is much more complicated. Do you calculate this value by hand when using cls with forest?

I mean, in terms of parallel trees and classes, the model slicing is as (or more) complicated.  You still have to do the calculation to get the correct tree count
Any additional lines of code required is point of violation of scikit-learn API.  One should be able to just 1) construct 2) fit 3) predict, not have to slice etc.  It at most should be an argument passed to fit or predict.  Else the whole purpose of scikit-learn API is lost.  There's a reason scikit-learn API is popular, it's simple and clean/understandable without arbitrary extra steps.  This is also why CUML etc. try to conform.
As for how to calculate, the early stopping model returns best_ntree_limit, so there is nothing to compute.  But this is the same as (best_iteration + 1 ) * num_parallel_tree if one needed to know.  Multi-class or not does not enter the equation ever.  See the tests, e.g. like in test_training_continuation you wrote :)

the proposal is to (by default) always have predict use the best_iteration result, so user gets the optimal early stopping result by default, that is a good proposal.

So far that's the default for single node scikit interface. Just making the alignment here.

I see yes, I was unsure about that.  I see that is true for the non-dask API, but I don't see for the dask API.

But it has to be made uniform between dask and non-dask.

Of course. We are making progress on the unification. The 2 interfaces are now much more closer than a few months before.

Also, then one should avoid needing to have to pass ntree_limit=0 for GLM, which is a pretty silly requirement.

Agree.

This is even further far away from any sklearn API and has all the same complications w.r.t. tree counts as ntree_limit.

Really? To me ntree_limit to me is much more complicated. Do you calculate this value by hand when using cls with forest?

I mean, in terms of parallel trees and classes, the model slicing is as (or more) complicated.  You still have to do the calculation to get the correct tree count
Any additional lines of code required is point of violation of scikit-learn API.  One should be able to just 1) construct 2) fit 3) predict, not have to slice etc.  It at most should be an argument passed to fit or predict.  Else the whole purpose of scikit-learn API is lost.  There's a reason scikit-learn API is popular, it's simple and clean/understandable without arbitrary extra steps.  This is also why CUML etc. try to conform.
As for how to calculate, the early stopping model returns best_ntree_limit, so there is nothing to compute.  But this is the same as (best_iteration + 1 ) * num_parallel_tree if one needed to know.  Multi-class or not does not enter the equation ever.  See the tests, e.g. like in test_training_continuation you wrote :)

the proposal is to (by default) always have predict use the best_iteration result, so user gets the optimal early stopping result by default, that is a good proposal.

So far that's the default for single node scikit interface. Just making the alignment here.

I see yes, I was unsure about that.  I see that is true for the non-dask API, but I don't see for the dask API.

But it has to be made uniform between dask and non-dask.

Of course. We are making progress on the unification. The 2 interfaces are now much more closer than a few months before.

Also, then one should avoid needing to have to pass ntree_limit=0 for GLM, which is a pretty silly requirement.

Agree.

For multi class, the ntree limit is:
(best_iteration + 1 ) * num_parallel_tree * num_class
Model slicing is one can use best_iteration for whatever model.  No extra calculation is required.so you can forget about the above equation.
I think I can see the confusion.  My point is not about model slicing, rather it's about using best_iteration instead of best_ntree_limit.   Currently we have the ntree_limit in skl interface predict function, the end result is it will be replaced by niter_limit(maybe a better name but hopefully I can get it clear for this discussion).  Model slicing will be the underlying mechanism for implementing it.  I mentioned it so that you can use it for now before we make the change.

For multi class, the ntree limit is:
(best_iteration + 1 ) * num_parallel_tree * num_class
Model slicing is one can use best_iteration for whatever model.  No extra calculation is required.so you can forget about the above equation.
I think I can see the confusion.  My point is not about model slicing, rather it's about using best_iteration instead of best_ntree_limit.   Currently we have the ntree_limit in skl interface predict function, the end result is it will be replaced by niter_limit(maybe a better name but hopefully I can get it clear for this discussion).  Model slicing will be the underlying mechanism for implementing it.  I mentioned it so that you can use it for now before we make the change.

@hcho3

Does it mean that pred_contrib is not available on Dask?

For skl dask, no.  I don't think right now skl interface has support for it.  Single node or distributed.

@hcho3

Does it mean that pred_contrib is not available on Dask?

For skl dask, no.  I don't think right now skl interface has support for it.  Single node or distributed.

For multi class (not binary), the ntree limit is:
(best_iteration + 1 ) * num_parallel_tree * num_class
Model slicing is one can use best_iteration for whatever model. No extra calculation is required.so you can forget about the above equation.
I think I can see the confusion. My point is not about model slicing, rather it's about using best_iteration instead of best_ntree_limit. Currently we have the ntree_limit in skl interface predict function, the end result is it will be replaced by niter_limit(maybe a better name but hopefully I can get it clear for this discussion). Model slicing will be the underlying mechanism for implementing it. I mentioned it so that you can use it for now before we make the change.

https://github.com/dmlc/xgboost/blob/24b87fc18a04d4aade26e058c6d21066978e98b5/tests/python/test_training_continuation.py#L123-L137
Is this testing code incorrect?

For multi class (not binary), the ntree limit is:
(best_iteration + 1 ) * num_parallel_tree * num_class
Model slicing is one can use best_iteration for whatever model. No extra calculation is required.so you can forget about the above equation.
I think I can see the confusion. My point is not about model slicing, rather it's about using best_iteration instead of best_ntree_limit. Currently we have the ntree_limit in skl interface predict function, the end result is it will be replaced by niter_limit(maybe a better name but hopefully I can get it clear for this discussion). Model slicing will be the underlying mechanism for implementing it. I mentioned it so that you can use it for now before we make the change.

https://github.com/dmlc/xgboost/blob/24b87fc18a04d4aade26e058c6d21066978e98b5/tests/python/test_training_continuation.py#L123-L137
Is this testing code incorrect?

They are wrong.... Tracking it down the bug dates back to 2016 or maybe even earlier.  ntree_limit is incorrectly set.

They are wrong.... Tracking it down the bug dates back to 2016 or maybe even earlier.  ntree_limit is incorrectly set.

Closing in favor of https://github.com/dmlc/xgboost/pull/6638 .

Closing in favor of https://github.com/dmlc/xgboost/pull/6638 .


bug: deterministic channel for multi-chain channel support

Overview
Make the channel creation determinstic, not based on the which spin ups first.

multiple chains: multiple relayer
multiple chains: single relayer


support for .tpp

Would it be possible to also support the .tpp extension for template files? I often have projects where template implementations are in a .tpp (which is included in the .h/.hpp at its end). Sometimes there is also a .hpp, .tpp and .cpp, but I would be fine, if switching would be tpp->hpp->cpp->hpp and in the case where there is only a .tpp and a .hpp it would do what one would expect.
Otherwise this extension is great and my new a.vim! :)

While I think about it: it would be nice, if the extensions were simply taken from the config like
    extension_preferences_table = {
        c = {h = 2, hpp = 1},
        h = {c = 2, cpp = 1},
        cpp = {hpp = 2, h = 1},
        hpp = {cpp = 2, c = 2},
        tpp = {hpp = 2, h = 1},
        inl = {cpp = 3, hpp = 2, h = 1},
        cu = {cuh = 3, hpp = 2, h = 1},
        cuh = {cu = 1}
    },

After some digging I found that my request is invalid, as this is already implemented.
As a reference for others that might hit the same bump:
This plugin gets auto-loaded for "filetype" "c" and "cpp", Therefore the command is not available in a buffer of filetype "cuda". "tpp" is not a recognized extension for NeoVim by default. To make .tpp become "filetype=cpp" add
vim.api.nvim_create_autocmd({"BufRead", "BufNewFile"}, {
    pattern = "*.tpp",
    callback = function() vim.bo.filetype = "cpp" end
})

somewhere in your main config. Now, to make this plugin also work with CUDA files this is the config (for Lazy) that I use:
return {
    {
        "jakemason/ouroboros.nvim",
        dependencies = {
            "nvim-lua/plenary.nvim",
        },
        config = function()
            -- these are the defaults, customize as desired
            require('ouroboros').setup({
                extension_preferences_table = {
                    c = {h = 2, hpp = 1},
                    h = {c = 4, cpp = 3, cu = 2, tpp = 1},
                    cpp = {hpp = 2, h = 1},
                    hpp = {cpp = 2, c = 3, tpp = 1},
                    tpp = {hpp = 2, h = 1},
                    inl = {cpp = 3, hpp = 2, h = 1},
                    cu = {cuh = 3, hpp = 2, h = 1},
                    cuh = {cu = 1}
                },
                -- if this is true and the matching file is already open in a pane, we'll
                -- switch to that pane instead of opening it in the current buffer
                switch_to_open_pane_if_possible = false,
            })
            vim.api.nvim_create_user_command('A', function()
                vim.cmd('Ouroboros')
            end, {})
            vim.api.nvim_create_autocmd("FileType", {
                pattern = "cuda",
                callback = function()
                    vim.api.nvim_buf_create_user_command(
                        0,
                        'Ouroboros',
                        function()
                            require('ouroboros').switch()
                        end,
                        {nargs = 0}
                    )
                end,
            })
        end
    },
}

This makes the command available for CUDA files as well (and defines :A) as a shortcut command.

Sorry for the late response, I've been super busy. I'm glad you got it figured out and saw that we now support this! I'll update the README to better reflect this functionality. Thank you very much for raising the issue!

No worries. Thanks for the reply! Feel free to copy&paste anything you might find useful.


Unable to reach OpsMan BOSH director

We attempted to add an external IP to our OpsMan BOSH Director (which by default has only a private IP) so that our Concourse workers could reach it. Unfortunately, the Director's CA cert will reject connections made to this IP:

Performing request GET 'https://35.xxx.xxx.xxx:25555/info': Performing GET request: Retry: Get https://35.xxx.xxx.xxx:25555/info: x509: certificate is valid for <IP_ADDRESS>, <IP_ADDRESS>, not 35.xxx.xxx.xxx

Ideally, the task would be able to set up an SSH tunnel (using e.g. sshuttle) to transparently proxy requests made to the OpsMan Director.
Thanks :)

sshuttle could be an option indeed (I'm a big fan of it). A property to set a value of BOSH_ALL_PROXY could be another option. I'll think about it. Thanks!

Am am also attempting this without deploying an external concourse worker to inside the customer plane PCF, BOSH_ALL_PROXY is definitely the preferred and best method rather than sshuttle

I started work on this, but external worker is still the best approach until UAAC supports SOCKS5

We ran into this issue recently and put in a PR to add jumpbox support for the bosh-creds-resource.  It exports the BOSH_ALL_PROXY into the bosh2_commandline_credentials and adds the jumpbox values to both of the jsons.  If left empty, they should be omitted by the bosh-deployment-resource

Thanks Mike! I have merged that PR and pushed mkuratczyk/bosh-creds-resource:1.5. However, unless I'm missing something, it doesn't solve this pipeline's problem completely because the first job still can't connect to the BOSH Director using uaac. What was your solution to this step?

Sorry about that.  I guess I did not read all the way through.  We had created the client ourselves.

One option that I recently picked up again is port forwarding through an ssh tunnel.  You can still specify the http protocol like you would any other request.  So theoretically, if you bind a port with the ssh tunnel of opsman against the director on port 8443 (with any port of your choosing), you can proxy through the bind.  I put a gist out there that uses two docker images for proxying the http server through the ssh server as a small demonstration.  Not sure if this could potentially get around uaac not allowing socks protocol, but I thought I would throw it out here.


add INetworkClient

I have similar branches for INetworkServer and INetworkManager. If this is accepted I will open PRs for those also.


Get size of char after normalization

Related to https://github.com/meilisearch/MeiliSearch/issues/1480

Fixed by #59


At least one parameter to parse_form is not actually None safe

Traceback (most recent call last):
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/webapp/server.py", line 7555, in index
    interview.assemble(user_dict, interview_status, old_user_dict, force_question=special_question)
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/base/parse.py", line 8032, in assemble
    raise the_error
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/base/parse.py", line 7821, in assemble
    question_result = self.askfor(missingVariable, user_dict, old_user_dict, interview_status, seeking=interview_status.seeking, follow_mc=follow_mc, seeking_question=seeking_question)
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/base/parse.py", line 8480, in askfor
    exec_with_trap(question, user_dict)
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/base/parse.py", line 9086, in exec_with_trap
    exec(the_question.compute, the_dict)
  File "<code block>", line 1, in <module>
  File "/usr/share/docassemble/local3.8/lib/python3.8/site-packages/formfyxer/lit_explorer.py", line 401, in parse_form
    title = reCase(re.search("(.*)\n",text).group(1).strip())
AttributeError: 'NoneType' object has no attribute 'group'

More specifically--looks like issue is when re.search() has no matches

Fixed in the notebook


Feature/multiple preview

Feature
Allow the use of wildcards to display more than one image.
Examples
pic -p iterm ~/Pictures/*.jpg

to display all .jpg images inside the Pictures directory.
pic ./*/*.png

to display all .png images inside any directory with depth 1 relative to pwd.
TODO

[ ] Display all image files if a directory is selected in path (Probably another PR)
[ ] Allow grid display of images (Another PR, big changes)

That's great work, how do you think this should work when specifying arguments as --xand --y ? Won't all images get over themselves ?

Just tested, -x does not affect as it just moves all images by an offset. -y does make the images overlap each other.
What do you think would be the intended result here?
For example, if I pass the flag -y 3, what should happen?

One possible thing is to make the -y flag useless if more than one image is requested.
Other is to make the -y flag a "spacing" flag between images (This would be harder imho)

Sounds good to me, for kitty protocol to work as you intend we would need to use different IDs, or we could use the uppercase I flag if I am not wrong, I know a bit about it so I should be able to deal with it

I was thinking of doing that with caching, we could store tempfiles in some repertory like .cache/pic or something, and not delete them so that we don't need to open the image each time (idk how we could name them to keep them unique and yet remember them), atm the tempfile is immediately being deleted by Kitty because it is not required anymore (Kitty loads the image in its memory, but if it's loaded without id it doesn't remain, there is a flag we can use so that it loads it with a random available id, see here

It would be great if we could keep track of image ids, much more efficient, it's so fast once the image is loaded

Maybe we remove the option to set IDs manually and set up a hash the image path + unique options (What I mean by this is that I don't know if kitty sets up width and height of the image after loading from memory or it saves it to memory with some width and height) so if I run pic with the same arguments as before, kitty just loads the hashed ID.

The power of kitty is that it once an image is loaded it can display it anywhere and resize it super quickly, the image is not stored with a fixed memory (kitty is the protocol I developed the most because I was only using this one in the beginning).
On the one hand your idea really seems good if we can figure a way to implement it, it could make kitty even better if well done.
But on the other hand the problem is the fixed options, with kitty (you can try) it will make it less efficient than loading and then displaying.
I already had some thought about that but I didn't want to make some shenanigans implementations... It's up to your imagination as of now, I like your ideas overall, and I will also tell you if something displease me, so try everything you have in mind we are not open source for nothing ;)

TODO

[ ] Create cache dir if it does not exist
[ ] When receiving a path, check if it is cached (Do nothing for the moment)
[ ] After using kitty protocol, create a file to create hashmap of FILE_PATH:ID
[ ] If using kitty and file is cached, load the image with the ID
[ ] After displaying image with kitty, modify cache dir with the image_path and modify file with NEW_PATH:NEW_ID
[ ] Somehow make this work?

I'm only wondering how we will get image ids, other than that it seems perfect to me, that's likely the way I would do it, but I couldn't figure a way to remember ids

For what I read in the link you sent, if we use the parameter I for kitty, it will reply with a valid i parameter and use that one to store the image. We need a way to get the response from kitty and store that i value as the ID.

For what I read in the link you sent, if we use the parameter I for kitty, it will reply with a valid i parameter and use that one to store the image. We need a way to get the response from kitty and store that i value as the ID.

Getting that id should be doable, look at the method to check if kitty protocol is supported, it should be approximately the same

To where  this is going, I think we should make cache feature in another pr. I'm going to try to get kitty to display multiple files quick ~and dirty~ and then move to another branch, because this feature is almost done. :)

Let's do this I will merge it when it's done, I should try to implement GIFs with kitty one day too, welp we will clean that later.

Sadly, I don't seem to be able to display more than one image at a time with kitty protocol...

Sadly, I don't seem to be able to display more than one image at a time with kitty protocol...

That's weird, I will look into it tomorrow and give it a try after school!

@Maratripa did you try to use the I parameter, I tried it quickly and it seems to work fine for me, we doesn't care about the id it returns for now but it's enough to display several images, isn't it ?


Update README with good instructions for getting lsp-julia up and running

https://discourse.julialang.org/t/using-lsp-julia/20085
This involves figuring out what in my setup is required for this to work and what isn't.

I think the setup should be much easier now.


Substitute models from Tensorflow or other platforms

Is that possible to train my own Resnet face cognition model on Tensorflow, then transfer it into a dat file and just substitute the current "dlib_face_recognition_resnet_model_v1.dat" model?
Is that possible to let the constructor load the new model instead of the pretrained model then?
If not, is there any ways I can adjust the parameters or improve the accuracy of the pretrained face recognition model?

See: #469
You don't have to use the pre-trained models. You can use dlib to train too.


feat: added qrcode for easier mobile access

Figured it would be a nicer UX to have a QR code to scan for when you're trying to open the site on mobile.

Changes:

added new dependency qrcode-terminal that handles the generation of the QR code

Can I request an admin, or anyone with write access, merge this PR? I can't merge it.


Name change

I've wondering why the latest version of a package (v<IP_ADDRESS>) on github (https://github.com/TheHydrophobicityProject/HydrophobicityExplorer) has updated on pypi.org (https://pypi.org/project/mhp/) but not on conda-forge (https://anaconda.org/scohenjanes/mhp). Perhaps, the url in setup.py needs updated to reflect the name change?

The last update on https://anaconda.org/scohenjanes/mhp/files occurred on 0ctober 18, 2022. Then, the name changed from mhp to Hydrophobicity Explorer on October 31, 2022. Since then, updates are not occurring. I'm going to change the url in setup.py and see if that helps.

It's actually not to do with the name--updates for pip and conda are handled a little differently, and there's currently an issue with building the Conda package.
I think updates to the pip package are handled manually by Sander running pypi_publish.sh using his credentials.
There's a GitHub Action configured to update the Conda package but it is set to run only on every GitHub release version (a good
idea to avoid updating the package on every commit), and we haven't published any GitHub release versions so it has never run.
I'm pretty sure the reason Sander hasn't run a release manually like he did for the pip package is the issues he described in #13 with the build taking a ridiculously long time and installing conflicting versions of packages--even if we can run the packaging workflow, we really don't want those kinds of issues making it into a released version.

It would be good to let GitHub actions handle this for us, because I don't imagine many major packages do manual packaging the way I have (though it's what all the tutorials I've seen use 😀)
I just opened another PR #15 that changed the build recipe for conda significantly. In short, the dependencies are no longer bundled in with our package. This reduces the file size from ~100 MB to ~30 kB. Strangely, it does not reduce the build time, but at least I was able to get the output package to work when installing the tar.bz2 file directly, and by installing from the scohenjanes channel on anaconda.

Also @mathersrob because the main branch was so many commits ahead of the name change branch, I updated it from the current main branch, so the merge can be completed without conflicts.
This would just cause the current PR I have open #15 to have conflicts, but that can be dealt with easily enough.

I updated it from the current main branch, so the merge can be completed without conflicts.

Upon review, I accepted the wrong changes for certain files so I am currently correcting that.


Return null on undefined untrustedMarkup

There are scenarios in which untrustedMarkup can be undefined (such as when splicing content into an array that is being repeated in a template) and exceptions are thrown. This change ensures a value of undefined returns null just like a value of null does.

@marknutter thanks for your contribution!  Couple things and we'll merge this ASAP:

please sign the CLA
please squash your commits and reword the commit message to follow the guidelines here.  This allows us to automate the creation of our release notes using the commit messages.

Thanks!

@jdanyow Signed the CLA and squashed the commits with a properly formatted commit message. Let me know if I need to do anything else and thanks for guiding me through your process!

CLA Confirmed :)

Takin' it back to the old school there ;)

Somebody set up us the bomb.

LOL :laughing:


fixes for ckeditor

This fixes the CKEditor issue

Does not fix everything, ckeditor CDNs still an issue for some plugins (like link)


Update docs for IP access lists

Codecov Report
Base: 40.73% // Head: 40.73% // No change to project coverage :thumbsup:

Coverage data is based on head (4140d08) compared to base (a88ca4d).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #282   +/-   ##
=======================================
  Coverage   40.73%   40.73%
=======================================
  Files          45       45
  Lines        2627     2627
=======================================
  Hits         1070     1070
  Misses       1475     1475
  Partials       82       82

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


CORE-17202 - change default smoketest target to Combined Worker

Currently the default smoketest target is Kubernetes.
This PR, along with the pipeline change, flips this default so that it is Combined Worker, which should reduce friction when running the tests in a local environment.

E2E tests against K8s:
[2023-09-21T13:37:18.686Z] Running tests against Kubernetes cluster
[2023-09-21T13:37:18.686Z] Crypto worker url: http://localhost:7001/
[2023-09-21T13:37:18.686Z] REST worker url: http://localhost:7002/
[2023-09-21T13:37:18.686Z] Flow worker url: http://localhost:7003/
[2023-09-21T13:37:18.686Z] Flow Mapper worker url: http://localhost:7004/
[2023-09-21T13:37:18.686Z] Verification worker url: http://localhost:7005/
[2023-09-21T13:37:18.686Z] DB worker url: http://localhost:7006/
[2023-09-21T13:37:18.686Z] Persistence worker url: http://localhost:7007/

against combined worker:
[2023-09-21T13:36:11.460Z] Running tests against combined worker
[2023-09-21T13:36:11.460Z] Crypto worker url: http://localhost:7004/
[2023-09-21T13:36:11.460Z] REST worker url: http://localhost:7004/
[2023-09-21T13:36:11.460Z] Flow worker url: http://localhost:7004/
[2023-09-21T13:36:11.460Z] Flow Mapper worker url: http://localhost:7004/
[2023-09-21T13:36:11.460Z] Verification worker url: http://localhost:7004/
[2023-09-21T13:36:11.460Z] DB worker url: http://localhost:7004/
[2023-09-21T13:36:11.460Z] Persistence worker url: http://localhost:7004/


Create and view resources from Unlisted Plan

This allows a resource to be created from an unlisted plan
The way we were querying plan details on create, list and view was incompatible with unlisted plans
This also fixes the broken resize command
And an issue with the Title Name swap in the update command
Relates manifoldco/engineering#3248

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Modify X.U.ClickableWorkspaces to work with X.A.DynamicWorkspaces

Problem Description
I am wondering if it is necessary or useful to modify X.U.ClickableWorkspaces to work with X.A.DynamicWorkspaces.
For single monitor, I think the change is quite simple. I would be happy to open a PR if it is considered useful.
After some reading, the only limitation here comes from getWsIndex.
-- | Lookup the index of a workspace id in the user's config, return Nothing
-- if that workspace does not exist in the config.
getWsIndex :: X (WorkspaceId -> Maybe Int)
getWsIndex = do
    spaces <- asks (workspaces . config)
    return $ flip elemIndex spaces

It only uses predefined workspaces.
I think for single monitor, a minor change would be good enough.
Checklist

[X] I've read CONTRIBUTING.md

[ ] I tested my configuration with xmonad-testing

I find out that getSortByIndex from X.U.WorkspaceCompare do address the issue. However, the issue exists in X.U.ClickableWorkspaces

Can you elaborate how exactly did you solve the issue using getSortByIndex? I have a rough idea what the problem is but if you have a working fix for ClickableWorkspaces that would be marvellous.
(I don't think I'll have time to fix this myself this week.)

So when the newly added workspace is passed to getWsIndex, it will return a Nothing. getSortByIndex manages this by using compare functions. So import a getSortByIndex will be helpful here.

Closed via #512


Add warning for enum contracts that don't pay you total collateral

As per @Christewart request

related to: https://github.com/bitcoin-s/bitcoin-s/issues/4070


metric.compute() hangs when using DDP with multiple GPUs

Bug description
I'm using the default Accuracy metric (though it appears to be true for any metric), and calling metric.compute() hangs after the first epoch and never resolves (ran it overnight, never progressed). It seems, as per some print() statements, that the issue is only with metric computation after training epoch ends, not after validation epoch ends. Issue does not happen for when using only 1 gpu or cpu. It is also agnostic of how large the dataset is, I tried with a dataset having only the first 2 batches and got the same result. I see there's another relevant issue (#5930) from 3 years ago, but has no solution (just says to update version and make a new issue).
What version are you seeing the problem on?
v2.4
How to reproduce the bug
In Model(pl.LightningModule).init(self, splits, #more args):
# ...
# without "_metrics" suffix, keys conflicts with nn.ModuleDict, Adding split prefix as well, but not sure if that is needed
 self.split_metrics = nn.ModuleDict({f'{split}_metrics': nn.ModuleDict({f'{split}_{name.replace(".","")}': metric for name, metric in {
    'acc': torchmetrics.Accuracy(task='binary'),
    # ... more metrics
}.items()}) for split in splits})
# ...

(Note: I considered using a metric collection, but some of my metrics need different inputs and I couldn't figure out how to account for that)
In Model.(pl.LightningModule)._step(self, batch, batch_idx, *, split, **kwargs):
# ...
for name, metric in self.split_metrics[f'{split}_metrics'].items():
    metric.update(y_pred_prob if needs_probability(name) else y_pred_label, y)
# ...

Relevant overloads in Model(pl.LightningModule):
def training_step(self, *args, **kwargs):
    return self._step(*args, **kwargs, split='train')

def validation_step(self, *args, **kwargs):
    return self._step(*args, **kwargs, split='val')

def on_train_epoch_end(self, *args, **kwargs):
    self._on_epoch_end(*args, **kwargs, split='train')

def on_validation_epoch_end(self, *args, **kwargs):
    self._on_epoch_end(*args, **kwargs, split='val')

Error messages and logs
I put some print statements that show it hangs for train_acc. This is with 2 GPUs.
Sanity Checking DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.80it/s][0] VALIDATION EPOCH END
[0] ENTERING _on_epoch_end :: val
[0] log dict initialized :: val
[0] now in loop for val_acc :: val
[1] VALIDATION EPOCH END
[1] ENTERING _on_epoch_end :: val
[1] log dict initialized :: val
[1] now in loop for val_acc :: val
[1] computed val_acc :: val
[1] logged val_acc :: val
[1] reset val_acc :: val
[0] computed val_acc :: val
[0] logged val_acc :: val
[0] reset val_acc :: val
Epoch 0:   0%|                                                                                | 0/2 [00:00<?, ?it/s][rank0]:[W823 21:27:11.992236756 reducer.cpp:1400] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[rank1]:[W823 21:27:11.034518963 reducer.cpp:1400] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
Epoch 0: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.34it/s, v_num=26[0] VALIDATION EPOCH END: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  5.13it/s]
[1] VALIDATION EPOCH END
[0] ENTERING _on_epoch_end :: val
[1] ENTERING _on_epoch_end :: val
[0] log dict initialized :: val
[1] log dict initialized :: val
[0] now in loop for val_acc :: val
[1] now in loop for val_acc :: val
[1] computed val_acc :: val
[0] computed val_acc :: val
[1] logged val_acc :: val
[0] logged val_acc :: val
[1] reset val_acc :: val
[0] reset val_acc :: val
Epoch 0: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00,  0.96it/s, v_num=26, loss/val=0.172][1] TRAINING EPOCH END
[1] ENTERING _on_epoch_end :: train
[1] log dict initialized :: train
[1] now in loop for train_acc :: train

Environment

Current environment
#- PyTorch Lightning Version (e.g., 2.4.0): 2.4.0
#- PyTorch Version (e.g., 2.4): 2.4.0
#- TorchMetrics Version: 1.4.1
#- Python version (e.g., 3.12): 3.11.9
#- OS (e.g., Linux): Linux
#- CUDA/cuDNN version: 12.1
#- GPU models and configuration: 8 x Tesla V100-SXM2-16GB
#- How you installed Lightning(`conda`, `pip`, source): pip

More info
No response

@manavkulshrestha could you provide how you log the metrics in training and validation?

Sure! I'm doing it in the ._step function:
#...
log_dict = {'step': self.current_epoch, f'loss/{split}': loss}
for name, metric in self.split_metrics[f'{split}_metrics'].items():
    metric.update(y_pred_prob if needs_probability(name) else y_pred_label, y)
    log_dict[name.replace('_','/')] = metric

self.log_dict(log_dict, on_step=False, on_epoch=True, prog_bar=False, logger=True, sync_dist=True)
#...

That looks fine to me, nothing that should make the process hang. Could you also share your implementation of self._on_epoch_end(*args, **kwargs, split='train') since that is where the code is hanging?

Hi, this is the code for that:
def _on_epoch_end(self, *, split:str):
        for metric in self.split_metrics[{split}_metrics'].values():
            metric.reset()
...
def on_train_epoch_end(self, *args, **kwargs):
    self._on_epoch_end(*args, **kwargs, split='train')

def on_validation_epoch_end(self, *args, **kwargs):
    self._on_epoch_end(*args, **kwargs, split='val')

def on_test_epoch_end(self, *args, **kwargs):
    self._on_epoch_end(*args, **kwargs, split='test')

are you still facing this issue @manavkulshrestha?
Found this thread because I encountered too, in my case I was only doing evaluation on the master process. Following this answer in a similar issue, it seems this is what causes the problem, I changed to calling it on all processes and the issue is now gone.
Hope this helps!


Docker-compose up working properly

The reverse-geoloc, app and game-creator images have been pushed to the
Docker Hub.
Also since docker-compose version 2 (specified in the .yml file), you are not forced to use the link directive to have two services talk to each other.

Awesome. Thanks!


Hostresolver: Treat AAAA queries as A queries when IPv6 is disabled

A "correct" answer would be to set handled = true and return a NODATA response.
Unfortunately some older resolvers use a slow random source to set the transaction id.
This creates a problem on M1 computers, which are too fast for that implementation:
Both the A and AAAA queries might end up with the same id. Returning NODATA for AAAA is faster, so would arrive first, and be treated as the response to the A query.
To avoid this, we will treat an AAAA query as an A query when IPv6 has been disabled.
This way it is either a valid response for an A query, or the A records will be discarded by a genuine AAAA query, resulting in the desired NODATA response.
Packet capture with the old implementation:
    1   0.000000 <IP_ADDRESS> → <IP_ADDRESS>  DNS 81 Standard query 0x0edd A mirrorlist.centos.org
    2   0.000020 <IP_ADDRESS> → <IP_ADDRESS>  DNS 81 Standard query 0x0edd AAAA mirrorlist.centos.org
    3   0.000288  <IP_ADDRESS> → <IP_ADDRESS> DNS 81 Standard query response 0x0edd AAAA mirrorlist.centos.org
    4   0.003276  <IP_ADDRESS> → <IP_ADDRESS> DNS 303 Standard query response 0x0edd A mirrorlist.centos.org A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS>

You can see both queries use the same transaction id 0x0edd, so the NODATA response in frame 3 is treated as the answer to the query in frame 1. Getting a second answer in frame 4 with different content is then considered an error:
curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error"

With the changes in the PR the packet capture looks like this:
    1   0.000000 <IP_ADDRESS> → <IP_ADDRESS>  DNS 81 Standard query 0x8064 A mirrorlist.centos.org
    2   0.000011 <IP_ADDRESS> → <IP_ADDRESS>  DNS 81 Standard query 0x8064 AAAA mirrorlist.centos.org
    3   0.172135  <IP_ADDRESS> → <IP_ADDRESS> DNS 303 Standard query response 0x8064 A mirrorlist.centos.org A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS>
    4   0.172158  <IP_ADDRESS> → <IP_ADDRESS> DNS 303 Standard query response 0x8064 AAAA mirrorlist.centos.org A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS> A <IP_ADDRESS>

So the A and AAAA queries still use the same transaction id, but since both responses contain the same requested data, the query succeeds and no error is reported.
Fixes https://github.com/rancher-sandbox/rancher-desktop/issues/1788 (and probably a couple of other DNS issues on M1 machines)

Shoutout to fellow packet log connoisseurs @chancez, @jsoref, and @nino-k for their help and the 570 message discussion on Slack!  😄

Alright, I've built the latest version of lima and am putting it through the paces. I'll let you know how it fares in a few days when I have more data!

Unfortunately, I'm still encountering very high DNS latency in my simple test.


[webhook] OwnerReferences for vault-agent ConfigMap missing if pod has a name

Describe the bug:
I'm creating a standalone pod with a CtConfigMap annotation. After the pod deletion, the "vault-agent-config" configmap does not get deleted.
Expected behaviour:
The ConfigMap should be garbage collected.
Steps to reproduce the bug:

Create a pod with a Consul-Template configMap configuration
Delete the pod

Additional context:
It seems that in the vault-agent-config configmap created by the webhook for the pod the ownerReferences object is missing, therefore there's no link for the K8s garbage collector.
According to https://github.com/banzaicloud/bank-vaults/blob/master/cmd/vault-secrets-webhook/main.go#L479 the OwnerReferences object is retrieved from the pod only if the name is not set (I assume this is for when the pod is part of a [replica|daemon]Set). In my case, the pod name is set, hence the vault-agent-config map name is taken from there, but the OwnerReferences object is missing.
I think the handling of OwnerReferences object should be separate from the name handling. If pod does not have an OwnerReferences object, one should be created and it should reference the pod, so that the configmap will be removed once the pod is deleted.
/kind bug

Hi @marcoreni, does your Pod instance have any ownerReferences set? If not, we can't really help here, otherwise, this is a bug, which can be solved if the Pod has ownerReferences, but if not, we don't know the UID of the Pod in this phase so it would be hard to get it and update the configMap later on (adds too much state to the current code).

I totally missed the uid while sweeping the ownerReferences docs, sorry for that.
I assume there's no way to work around this, unless we find a way to remove the need of this configmap..
Has anyone considered the possibility to migrate from Consul Template to Vault Agent 1.3 (that introduced the capability of parsing Templates)? AFAIU, this would remove the need of Vault Agent init-container (at the cost of probably defining all configuration values inside the consul-template configmap object) so the additional vault-agent-configmap wouldn't be required anymore.. [ it would also probably solve https://github.com/banzaicloud/bank-vaults/issues/464, but that's a different story :) ]

Yes we definitely need to evaluate the new vault-agent feature! :-)
i have found a possible bug in the setting of the owner reference in the meantime which doesn't solve this issue entirely but tries to get the reference in all situations.


Remember to support *.gemspec files in Ruby-on-Rails projects

Remember to support *.gemspec files in Ruby-on-Rails projects. Many times you find these instead of Gemfile file.

I have thought about this, but the problem is that gemspec files are native Ruby so in addition to even just having to parse Ruby (which is a syntactically complex language), you have to account for the behaviour at runtime because you can have conditions (though I'm not sure if there is a realworld usecase for doing that in a gemspec file for listing a dependency).
The gemspec file also generally represents dependency constraints rather than exact versions - meaning you're probably not going to get accurate results unless you expand those constraints into every possible version (which could be all versions, and the detector doesn't know how to do this because that would require interacting with gem repos specifically).
Obviously right now an easy way to resolve this is to have a Gemfile that pulls from your gemspec:
# frozen_string_literal: true

source 'https://rubygems.org'

gemspec

and then check the resulting Gemfile.lock - personally I don't think that should be a problem, but if people really don't like having both a gemspec & Gemfile, then maybe a good middleground could be to have a Ruby script that generates a CSV for the detector based on the gemspec; that way you could use Bundler to expand the constraints (iirc there's even a method or two dedicated to this)

Thanks for nice response. You can close or keep around as a reminder.

pruning


Use NonMaxU32 for LineNumber

This will reduce their size in memory substantially when they're optional.

Line numbers aren't a major contributor to memory usage.


OCPBUGS-19842: Limit deletion webhook to openshift namespace

This PR changes the Cluster delete webhook, so that it will only prevent deletion when the cluster is in openshift-cluster-api namespace.

/jira refresh

/lgtm
/label acknowledge-critical-fixes-only
This only affects a tech preview feature and is not something we currently exercise in any release blocking or informing jobs


Update workflow to point back at rollmint

Overview
Moving the build to use the main of celestiaorg/ethermint which has been upgraded to a newer version of ethermint and uses rollmint instead of optimism as it should.

4. The triggers should be modified so we know this is working in a PR

I think jobs are skipped because this PR is from fork, and we disabled them because of credentials / permissions.

I've updated these to be a closer match to the celestia-node builds.

Is this issue still relevant after https://github.com/celestiaorg/ephemeral-cluster/pull/33

@joroshiba Is this issue still relevant after celestiaorg/ephemeral-cluster#33

Yes. This is updating to use a more up-to-date version of the Ethermint fork which uses cosmos v0.46.3, while existing fork uses v0.45.5.
The ephemeral cluster updates were more about the usability of the ephemeral cluster.

Stale PR since https://github.com/celestiaorg/ephemeral-cluster is archived.


OSError: output\vgg16\voc_2007_trainval+voc_2012_trainval\default\vgg16_faster_rcnn_iter_70000.ckpt.meta not found. Did you download the proper networks from our server and place them properly?

I have already trained the train.py, when I open demo.py and run, the error occurs:
OSError: output\vgg16\voc_2007_trainval+voc_2012_trainval\default\vgg16_faster_rcnn_iter_70000.ckpt.meta not found.
Did you download the proper networks from our server and place them properly?
do you know that?

How do you solve this problem？pls tell me.

make a new file output\vgg16\voc_2007_trainval+voc_2012_trainval\default\ by yourself

where did you make this new file? Pls tell me

where did you make this new file? Pls tell me

make a new file output\vgg16\voc_2007_trainval+voc_2012_trainval\default\ by yourself

I have place them properly. Thank you.


async version of the currentTree function

Thank you!!

@kaunteya I just realized that this API returns an optional Tree. That is not required, sop I'm going to modify the API. Just wanted to let you know in case it breaks anything.

Sure, Matt. No issues


Fixed ISO8601 Crashes

I'm getting a crash when parsing ISO dates of the form "2018-08-01" on iOS 12 using Xcode beta 6 on the simulator.
Simply adding this line to applicationdidFinishLaunching causes the crash:
let date = ISOParser.date(from: "2017-01-03")
The crash is a "String index is out of bounds" error inside the current() method.
Parsing longer ISO dates works as expected. I haven't updated the code to swift 4.2. Swift version is set to 4. SwiftDate is 5.0.7 and is installed with CocoaPods.
Parsing dateStrings like this has always worked fine on iOS 11 and earlier.

I have the same issue.

Apparently any deviation from the required ISO format is likely to cause this crash. I had a string in a unit test that was missing the trailing Z and that crashed. Obviously something has changed. These dates were accepted previously. If the intent is for them to not be accepted ISOParser should still not crash. It's supposed to return nil.
My workaround for my short dates "2018-08-01" has been to use a DateParser with the matching format.

This is not related to XCode 10 but it's a bug of the parser. Fixed it and added several other unit tests.

Hey @redent, I cherrypicked the fix onto latest 4.x and it fixed my issues. (link to commit above)

@malcommac Any update on whether or not this change could be PR'd into a 4.x branch? Our project relies heavily on SwiftDate and upgrading to 5.x is not something we can do right now.

@malcommac Similar to @MrMatthewDavis and @raudabaugh we'd like to ask this fix retrofitted to 4.x if at all possible - for reasons articulated by MrMatthewDavis.
SwiftDate 4.3.0 and ISO8601Parser with a date string composed of a year and a week number (e.g. "2019-W13") appears also to crash, please see an example below:
let date = ISO8601Parser.date(from: "2019-W13")?.date


error when during running on llff dataset

Thanks for your great job!
I encountered a bug when I test on llff dataset:
(sparf-env) juneyoung@juneyoung:~/Programmes/sparf$ python run_trainval.py joint_pose_nerf_training/llff sparf --scene=orchids --train_sub 3
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: /home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/lpips/weights/v0.1/vgg.pth
/home/juneyoung/Programmes/sparf/source/utils/colmap_initialization/../../../third_party/DenseMatching
Training:  joint_pose_nerf_training/llff/subset_3/orchids  sparf
Date: 27/09/2023
(creating new options file...)
[2023-09-27 19:46:02] [INFO] Command executed: run_trainval.py joint_pose_nerf_training/llff sparf --scene=orchids --train_sub 3
[2023-09-27 19:46:02] [INFO] Training:  joint_pose_nerf_training/llff/subset_3/orchids  sparf
Date: 27/09/2023
[2023-09-27 19:46:02] [INFO] Tensorboard is enabled. Write events to /home/juneyoung/Programmes/sparf/run_cache/joint_pose_nerf_training/llff/subset_3/orchids/sparf.
[2023-09-27 19:46:02] [INFO] Using Single-GPU mode.
[2023-09-27 19:46:02] [INFO] Using device cuda
[2023-09-27 19:46:02] [INFO] Setting seed 0
Setting up [LPIPS] perceptual loss: trunk [alex], v[0.1], spatial [off]
/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: /home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/lpips/weights/v0.1/alex.pth
[2023-09-27 19:46:03] [INFO] loading training data...
training dataset: llff
Train dataset llff has 3 samples
number of samples: 3
[2023-09-27 19:46:03] [INFO] loading test data...
eval dataset: llff
eval scenes: orchids
Val dataset llff has 4 samples
number of samples: 4
/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
[2023-09-27 19:46:03] [INFO] Depth type inverse
[2023-09-27 19:46:03] [INFO] depth range 1 to 0
[2023-09-27 19:46:03] [INFO] building networks...
/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading flow checkpoint from /home/juneyoung/Programmes/sparf/PDCNet_megadepth.pth.tar
[2023-09-27 19:46:04] [CRIT] Found 3/3 valid initial poses
[2023-09-27 19:46:04] [CRIT] All initial poses: error_R_before_align: 6.202, error_t_before_align: 0.258, error_R: nan, error_t: nan
[2023-09-27 19:46:04] [INFO] Creating NerF model for joint pose-NeRF training
Traceback (most recent call last):
  File "/home/juneyoung/Programmes/sparf/run_trainval.py", line 156, in <module>
    main()
  File "/home/juneyoung/Programmes/sparf/run_trainval.py", line 151, in main
    run_training(args.train_module, args.train_name, cudnn_benchmark=args.cudnn_benchmark, seed=args.seed,
  File "/home/juneyoung/Programmes/sparf/run_trainval.py", line 105, in run_training
    trainer = define_trainer(args=settings, settings_model=model_config,
  File "/home/juneyoung/Programmes/sparf/source/training/define_trainer.py", line 97, in define_trainer
    trainer = PoseAndNerfTrainerPerScene(args)
  File "/home/juneyoung/Programmes/sparf/source/training/joint_pose_nerf_trainer.py", line 414, in __init__
    super().__init__(opt)
  File "/home/juneyoung/Programmes/sparf/source/training/nerf_trainer.py", line 51, in __init__
    self.init_for_training(opt)
  File "/home/juneyoung/Programmes/sparf/source/training/nerf_trainer.py", line 60, in init_for_training
    self.build_networks(opt)
  File "/home/juneyoung/Programmes/sparf/source/training/joint_pose_nerf_trainer.py", line 489, in build_networks
    plotting_dict = self.visualize_poses\
  File "/home/juneyoung/Programmes/sparf/source/training/joint_pose_nerf_trainer.py", line 379, in visualize_poses
    return self.visualize_any_poses(opt, pose, pose_ref_, step, split=split)
  File "/home/juneyoung/Programmes/sparf/source/training/joint_pose_nerf_trainer.py", line 338, in visualize_any_poses
    pose_vis = util_vis.plot_save_poses(opt,fig,pose_aligned,pose_ref_w2c=pose_ref,ep=self.iteration)
  File "/home/juneyoung/Programmes/sparf/source/utils/vis_rendering.py", line 362, in plot_save_poses
    ax1.scatter(cam[i,5,0],cam[i,5,1],cam[i,5,2],color=c,s=40)
  File "/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/matplotlib/__init__.py", line 1465, in inner
    return func(ax, *map(sanitize_sequence, args), **kwargs)
  File "/home/juneyoung/anaconda3/envs/sparf-env/lib/python3.9/site-packages/mpl_toolkits/mplot3d/axes3d.py", line 2377, in scatter
    if kwargs.get('color', None):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Could you please provide some ideas to solve it?

@JunyuanDeng Just passing through. The error
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() happens when you check if a numpy array is true, which doesnt make sense. How can an array be True, one should convert array of bool to bool?
Now, you are getting error here if kwargs.get('color', None):, which means kwargs has value color which is a numpy array. This is causing the problem. But this code is of matplotlib.
The sparf code which calls matplotlib is:-
ax1.scatter(cam[i,5,0],cam[i,5,1],cam[i,5,2],color=c,s=40)
which looks fine.
Digging deeper into matplotlib and looks like a recent change broke this - https://github.com/matplotlib/matplotlib/issues/26821
Maybe downgrade or upgrade matplotlib.
P.S. @ PruneTruong - Nice work. Following this repo to have fun when I have bigger gpu. :)

@saurabheights  Thanks for your help! I will try it later!


Add cacti as a submodule

It seems more elegant to add cacti as a submodule for the following reasons:

it is easier to track what version of cacti that is used
If we want tweaked dependencies, a fork can be made with its own history
reduces size of the repository

Furthermore, currently the folder contains compiled binaries that I think were not intended to be pushed.
That said, an additional attention is needed when cloning the repository. Users would have to run git clone --recursive to clone the submodules when cloning the main repo.
Feel free to close the pull request if you deem the current method to be fine.

Unfortunately we had to modify some cacti files, so this isn't possible. In the future we will move to another memory modeling tool.


Editing single blank cell and applying to identical cells does not work affect other blank cells

Original author<EMAIL_ADDRESS>(November 05, 2010 17:15:12)
Click edit on a blank cell, enter something, and click Apply to identical cells. That does not affect other blank cells.
Original issue: http://code.google.com/p/google-refine/issues/detail?id=180

Just an update - I'm working on this and work in progress is at https://github.com/ostephens/OpenRefine/tree/mass-edit-fix
tl/dr fix #1631 is relatively straightforward, but I'm trying to make tests and fix #1632, #180 and #332 at the same time which is proving more complicated and taking more time

Fixed by #1642


Support for "Import failed"

This could apply to all *arr apps but the main one I get this issue with is Lidarr. Often I have downloads that fail to import due to XYZ reasons such as "track match too low" or "wrong album", etc. I was hoping to see if you could add an option to have Swaparr also detect these on its runs and blocklist + search again as it's very tiring to manually go through hundreds that popup everytime I click search all on my wanted list.
Thanks!

Hi @909oce,
There have been quite a lot of functional changes, it might also target your needs now. I am not quite certain though, so I'd like to ask for your feedback.
Please do reopen this issue if it's still relevant for you 😊
^ Thijmen


Release consistency

Making the release 18.1.2 on LINK marketplace exactly same as our version
(Only whitespaces different)

@maassenbas can we remove this file on both releases: metadata/site_import/sites/.DS_Store ?


improvement of cockroach dialect and some fixes

Hi @markbates, @stanislas-m, and all!
Since my recent PR for improving Cockroach dialect(#271) are not merged smoothly, (maybe affected by recent Github failure?) so I wrote it again against current development branch.
This includes:

support secure mode configuration with options without url. (can add any options)
fixed urlWithoutDb() can handle secure mode with options.
bugfix and clean up of connection_details.go.
added test cases for above changes.

Additionally, @eduDorus reported a bug on the cockroach dialect (https://github.com/gobuffalo/buffalo/issues/1417)

fixed TrucateAll() can support both v1 and v2 of Cockroachdb.
fixed test case for TrucateAll().

Please check this.

Also, Please check my proposal (#281) to enhance the structure of connection details, dialects, and connection. I think now is time to do it.

Thanks for the quick check!

Nice! Thanks!


[Search] Ensure the search for both artists and genres work

Genre search finished in commit

Artist search finished in pull request


FR: need push-to-talk mode

Would be useful to be able to configure a keyboard shortcut for push-to-talk like with mumble.

+1 to that

@aspiers, @kbaikov I have deployed a preview of this feature (developed by @sandr3j) at https://jangouts.tk
Can you please give it a try and say if it fits your expectations.

Awesome, thanks! We just tested this and it works great for some hotkeys (and I quite like the UI design), but not for others. For example, I tried using Super_R + U and for some weird reason it got incorrectly recognised as \.  Then I could press \ to successfully push-to-talk, but of course that conflicts with using \ for other purposes.
The other thing is that it will need persist the choice of hotkey between sessions - having to set it each time would be annoying.

I tried to reproduce, but for me super is always recorded as meta. @aspiers Which system/browser are you using?

To be precise, my Super_L is recorded as meta. I don't have a Super_R key.

Even after using xmodmap to remap my super key to Super_R, it is still being registered as meta by Jangouts.

Seems to be a general issue with the library we use (mousetrap). https://github.com/ccampbell/mousetrap/issues/81
So I would happily declare super to be unsupported.
Or maybe this other mousetrap alternative handles it better https://github.com/PolicyStat/combokeys... or maybe not.

I'm using Chromium 49 on openSUSE 13.2.  A quick glance at https://github.com/ccampbell/mousetrap/issues/81 seems to suggest that they are using keycodes instead of keysyms to detect which key was pressed?!  If so, that's incredibly broken (because any keycode can be remapped to any keysym) and so you should avoid that library at all costs.  But maybe I'm missing something.

And BTW I can't think of any good reason why some modifiers should be supported on Linux but not others.  They're supposed to be handled in the same way, as you can see via the modifier bitmap in xev.  Maybe it's because this is a cross-platform library and they're focusing on Windows/Mac :-/

Mousetrap is already used by AngularHotkeys, a component we are already using in other parts of Jangouts (which is not surprising, since Mousetrap is the most popular keystrokes library out there). Thus, it simply made sense to use it for the push-to-talk feature.
If we can find a (almost) drop-in replacement that doesn't interfere with Angular-Hotkeys, I'm fine with changing it now. But if it takes us more than a couple of minutes, I would rather consider the feature done for the time being (as soon as we implement persistence of the configuration) and would open a separate low-priority issue to replace Mousetrap.

@ancorgs Sounds like a good approach to me.  Thanks!

@aspiers @ancorgs Thanks for your feedback. I checked today chromium and firefox on openSUSE 13.1 and could verify the problem. There are even some browser differences. I think we can at least exclude the problematic keys. I will investigate what kind of workaround might be the best. Of course there is no problem with adding the hotkey to persistent configuration. BTW why not just make the space key default and offer an option to change the key? I'm not sure if it is clear that the hotkey is never triggered while typing in the chatbox.

I would not activate push-to-talk by default. For me it's more an additional goodie than a core feature. Of course it has a usecase (that's why it was requested, after all), but I don't think it's an intuitive thing that most videoconferencing users would want/expect to be active.
In other news, the 1pixel increase in the header is back. :-)

@aspiers @kbaikov FYI, jangouts 0.4.4 (push-to-talk without key combinations) deployed in the internal SUSE's jangouts.

Thanks!


Add examples from DeepMind paper (Learning explanatory rules from noisy data)

I've added the examples used in the paper by (Evans & Greffenstette, 2018). They are laid out in the paper in Appendix G. In most cases, it was a direct translation to code. In some cases, I filled in the blanks where details were unspecified.

Excellent, thank you. I will look at merging this change into the main repository at the weekend.
Kind regards,
Andrew

On 2 Apr 2019, at 19:43, Abhijeet Krishnan<EMAIL_ADDRESS>wrote:
I've added the examples used in the paper by (Evans & Greffenstette, 2018). They are laid out in the paper in Appendix G. In most cases, it was a direct translation to code. In some cases, I filled in the blanks where details were unspecified.
You can view, comment on, or merge this pull request online at:
https://github.com/metagol/metagol/pull/15
Commit Summary
• Add 3 examples
• Add more examples
File Changes
• A examples/adjacent-to-red.pl (36)
• A examples/grandparent.pl (41)
• A examples/graph-colouring.pl (35)
• A examples/graph-connectedness.pl (31)
• A examples/less-than.pl (37)
• A examples/member.pl (46)
• A examples/predecessor.pl (34)
• A examples/relatedness.pl (36)
• A examples/son.pl (42)
• A examples/undirected-edge.pl (25)
Patch Links:
• https://github.com/metagol/metagol/pull/15.patch
• https://github.com/metagol/metagol/pull/15.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I'm porting these new examples to the Logtalk port of Metagol. Found two minor issues with the original code. The member.pl and son.pl report singleton variables. In member.pl, R is a singleton in the clause:
metarule([P,Q,R],([P,A,B]:-[[Q,B,C],[P,A,C]])).

In son.pl, R is a singleton in the clause:
metarule([P,Q,R],([P,A]:-[[Q,A,B]])).

It seems fine to replace these singleton variables with the anonymous variable. Do you agree?

Hi Paulo,
Yes, those two variables can be changed to anonymous variable. Thank you for spotting them.
Kind regards,
Andrew

On 3 Apr 2019, at 14:57, Paulo Moura<EMAIL_ADDRESS>wrote:
I'm porting these new examples to the Logtalk port of Metagol. Found two minor issues with the original code. The member.pl and son.pl report singleton variables. In member.pl, R is a singleton in the clause:
metarule([P,Q,R],([P,A,B]:-[[Q,B,C],[P,A,C]])).
In son.pl, R is a singleton in the clause:
metarule([P,Q,R],([P,A]:-[[Q,A,B]])).
It seems fine to replace these singleton variables with the anonymous variable. Do you agree?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks for confirming. Do you have the BibTex record for the paper? I would like to add it to the port documentation and maybe also extract some unit tests from the paper if you can send me a PDF version.

Here is the bibtex record:
https://dblp.uni-trier.de/rec/bib2/journals/jair/EvansG18.bib
and paper:
https://arxiv.org/pdf/1711.04574.pdf

On 4 Apr 2019, at 11:24, Paulo Moura<EMAIL_ADDRESS>wrote:
Thanks for confirming. Do you have the BibTex record for the paper? I would like to add it to the port documentation and maybe also extract some unit tests from the paper if you can send me a PDF version.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks! Updated the port notes (https://github.com/LogtalkDotOrg/logtalk3/blob/master/ports/metagol/NOTES.md).


Manually restart service provisioned through forever-service?

Perhaps I am missing something, but how can I manually start or stop a service that was provisioned with forever-service, after it has been installed?
I am running on Ubuntu 14.04, and after installing the service (forever-service install myservice), if I don't use the --start option, I am unable to start it as it doesn't get added as a service on Ubuntu (so service myservice start fails saying myservice is unrecognized), and I also cannot manually start it with forever (also says that it is unrecognized).
Can someone point me in the right direction?

On my Ubuntu 14.04, I simply use:
sudo start <forever-service-name>   # to start the service
sudo stop <forever-service-name>    # to stop the service

whenever I want to start or stop any forever-service service.

Thanks, that helps. Looks like restart doesn't work if the service is stopped, but that's sufficient for me.


Test ExoPlayer

image has 3s delay when change surfaceholder  . -_- means to fullscreen or quit fullscreen will have 3s delay

What can be the cause?

https://github.com/lipangit/TestExoPlayer
There have two surfaceview click a surfaceview will play on the surfaceview. we will see image 1s-4s delay but the sound is continuous,

enter fullscreen exoplayer can not make video smooth


Immediate window debugger off by a line

Applies To

[ ] Notebooks (.ipynb files)
[X] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
Opened Immediate Window.
Created demo.py with the below code and you can see the debugger is off by what appears to be a line. If I remove the breakpoint on the whitespace line it won't break at all:

import matplotlib.pyplot as plt
fig, ax = plt.subplots()

fruits = ['apple', 'blueberry', 'cherry', 'orange']
counts = [40, 100, 30, 55]
bar_labels = ['red', 'blue', '_red', 'orange']
bar_colors = ['tab:red', 'tab:blue', 'tab:red', 'tab:orange']

ax.bar(fruits, counts, label=bar_labels, color=bar_colors)

ax.set_ylabel('fruit supply')
ax.set_title('Fruit supply by kind and color')
ax.legend(title='Fruit color')

plt.show()

VS Code Version
1.76.1
Jupyter Extension Version
v2023.2.1200692131
Jupyter logs
No response
Coding Language and Runtime Version
python 3.10.9
Language Extension Version (if applicable)
v2023.4.1
Anaconda Version (if applicable)
No response
Running Jupyter locally or remotely?
Local

Sorry, what did you do to start debugging that .py file? I think that we assume it has a # %% comment

I'm using shift enter with this setting:

Thanks, as a temp workaround you might try adding a cell comment to the top of the file

Thanks, as a temp workaround you might try adding a cell comment to the top of the file

Thanks! That works.

@jabbera Are you still able to repro this issue with the latest versions of VS code, Jupyter, Python and Python debugger extenssions?


Investigate style drop down list doesn't show styles

For Mines Operating Status data layer, the drop down style list appears white (with no styles to select from):

@steve9164 would you please have a quick look and let me know if this big issue to fix?


fix cheercard

fixes #22

Thanks @kaustav202 for the new PR. However, this does not match up with the requirements.
We need to remove the Share button from card "So that it won't appear in the downloaded image itself"
Looks like you currently removed the button entirely from the page.
We need the button so that the user can share the card; however, it does not need to be part of the card - otherwise, the button will also appear on the card. e.g. see this screenshot - the button also appears in the card (ideally, it should not be part of the shared image/card):

Take a look on PR for Issue #11  - https://github.com/hacktobered/hacktobered.com/pull/20
Or this file change - https://github.com/hacktobered/hacktobered.com/pull/20/files

Alright.. I'll change it


⚠ main has degraded performance

In 9647e74, main ($MAIN_API_URL) experienced degraded performance:

HTTP code: 200
Response time: 23204 ms

Resolved: main performance has improved in 3e85e44.


Could not remove an access from a file

When we share a file with one of our contacts, then we have a button for removing access.
Clicking on this button will occur an error with this text: No Provider with id "ocFederatedSharing" found

it is the same  with issue #7

This error occurs on line 33  of Share20OcsController.php.
I think this is a problem with registerProvider that not configured this provider.


2648: add detailed info about allocation batch job processes

Context
Add detailed info about bulk allocation jobs processing results.
Tickets:

Allocation release numbers
Allocation Bulk Upload should signal when all allocations have been processed

Changes proposed in this pull request

Add field :applied_allocation_delta to AllocationBatchJob to store the effective delta applied to a school allocation as part of the execution of the job.
Display details (jobs processed, aggregate allocation change) about the results of processing bulk allocation jobs.
Highlight school rows whose allocation delta does not match the applied one.
Added feature tests to bulk allocations upload and post-processing result view.

Guidance to review

 Allocation released numbers

 Allocation Bulk Upload should signal when all allocations have been processed


Show parent group in report

If a job group has a parent it will be shown as:
Parent Group / Job Group

(Where) are we generating a full report in the tests?
I updated all tests that were affected, but they only generated a list of jobgroups.
The job group title in the complete report (there is no jobgroup title in the report that only covers one jobgroup iirc) will be prefixed with parent groupname /  if the jobgroup has a parent.

tried it out manually, looks good, tests still work, coverage still 100%, good enough. thanks.


Selection styling

Rather than applying an overlay which obscures the things you're most interested in, try using saturation/desaturation/opacity to distinguish selected and unselected.

fixed by #41


Change variable "retryDelay" into  "RetryDelay" to stay the same in (tests/integration/helm/util.go)

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[X ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[ ] Does not have any user-facing changes.
Before:
const (
IstioNamespace      = "istio-system"
ReleasePrefix       = "istio-"
BaseChart           = "base"
DiscoveryChart      = "istio-discovery"
IngressGatewayChart = "istio-ingress"
EgressGatewayChart  = "istio-egress"
BaseReleaseName     = ReleasePrefix + BaseChart
IstiodReleaseName   = "istiod"
IngressReleaseName  = IngressGatewayChart
EgressReleaseName   = EgressGatewayChart
ControlChartsDir    = "istio-control"
GatewayChartsDir    = "gateways"
retryDelay          = 2 * time.Second
RetryTimeOut        = 5 * time.Minute
Timeout             = 2 * time.Minute
)
After:
const (
IstioNamespace      = "istio-system"
ReleasePrefix       = "istio-"
BaseChart           = "base"
DiscoveryChart      = "istio-discovery"
IngressGatewayChart = "istio-ingress"
EgressGatewayChart  = "istio-egress"
BaseReleaseName     = ReleasePrefix + BaseChart
IstiodReleaseName   = "istiod"
IngressReleaseName  = IngressGatewayChart
EgressReleaseName   = EgressGatewayChart
ControlChartsDir    = "istio-control"
GatewayChartsDir    = "gateways"
RetryDelay          = 2 * time.Second
RetryTimeOut        = 5 * time.Minute
Timeout             = 2 * time.Minute
)

Hi @zhucheng01. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

@zhucheng01: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
4672e29e1ccfeaf1f00e16f3bb95ba8bb777e6e9
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

You will want to add a fixes statement in the header or link this PR to your issue for the misspelling: #33277 if indeed this is the misspelling.
Also, to fix the release_notes failure (you can see the output by click in Details to the right of the test and then the Raw build-log.txt link). To set the label, I believe the bot will do that if you put an x in the [ ] in [ ] Does not have any user-facing changes.

/retest


Byte Commander: Blacklist hemp\Woil

Byte Commander requests the blacklist of the keyword hemp\Woil. See the Metasmoke search here

The linked regex search seems broken...? I get 21/21 tpu's with https://metasmoke.erwaysoftware.com/search?utf8=✓&body=Hemp+Oil&user_rep_direction=>%3D&user_reputation=0&commit=Search

Yeah, it's a known issue
https://github.com/Charcoal-SE/metasmoke/issues/258

!!/approve


I'm having trouble setting up the absolute paths for an non-default directory structure

I've been chipping away at the features and I've had success doing a lot of things with webpack.  One thing that has plagued me for sometime is setting up a directory structure where the conf, bower and node modules are stored in a directory that is a sibling of the directory where my custom modules are stored.
I'm receiving the following error
ERROR in Entry module not found: Error: Cannot resolve module coffee-loader in /usr/local/webprojects/webpack/app/modules
resolve module coffee-loader in /usr/local/webprojects/webpack/app/modules

Note that I was successfully working with my node and bower directories being in different places before.  Something about the absolute path situation is messing my project up, since it is not on the same branch.
My project is laid out like the following:
project
    build
        bower_components
        node_modules
        bower.json
        package.json
        webpack.config.js
    app
        modules (custom/proprietary commonjs modules)
        php
        view (Phalcon Volt partials)
        config (json configs so that socket.io and php can both connect)
        public
            index.php
    dist
        [working stuff will go here and will be fetched in the debug PHP template]

test might go inside of build or on the top level

I've set it up like this since PHP compilation will be a feature of this framework/scaffold

var path = require('path');
var webpack = require('webpack');
var console = require('console');

var resolveBowerPath = function(componentPath) {
    return path.join(__dirname, 'bower_components', componentPath);
};
var resolveNodePath = function(componentPath) {
    return path.join(__dirname, 'node_modules', componentPath);
};
var resolveModulePath = function(itemPath) {
    var appPath = path.resolve('../app');
    return path.join(appPath, 'modules', itemPath);
};

console.log(resolveBowerPath(''));
console.log(resolveNodePath(''));
console.log(resolveModulePath(''));
console.log(path.resolve('../app') + '/public/bundle');

module.exports = {
    cache: true,
    context: resolveModulePath(''),

    entry: {
      entryMain: './entryMain'
//      entryLogin: './entryLogin'
    },

    output: {
//        path: __dirname + '/public/bundle',
        path: path.resolve('../app') + '/public/bundle',
        publicPath: '/bundle/',
        filename: "[name].bundle.js",
        chunkFilename: '[id].bundle.js'
    },

    resolve: {
//        root: resolveModulePath(''),
        root: [resolveBowerPath(''), resolveNodePath(''), resolveModulePath('')],
        // tells webpack to query these directories for modules
//        modulesDirectories: ['bower_components', 'node_modules'],
        modulesDirectories: [resolveBowerPath(''), resolveNodePath(''), resolveModulePath('')],
//        modulesDirectories: ['./bower_path', './node_modules', resolveModulePath('')],

        alias: {
            lodash: resolveBowerPath('/lodash/dist/lodash'),
            jquery: resolveBowerPath('/jquery/dist/jquery'),
            angular: resolveBowerPath('/angular/angular'),
            angularRouter: resolveBowerPath('/angular-ui-router/release/angular-ui-router'),
            angularCookies: resolveBowerPath('/angular-cookies/angular-cookies'),
            angularResource: resolveBowerPath('/angular-resource/angular-resource'),
        },
        extensions: [
            '',
            '.js', '.coffee',
            '.html', '.jade',
            '.css', '.styl', '.scss', '.less'
        ]
    },
    plugins: [
        new webpack.ProvidePlugin({
            _: 'lodash',
        }),
        // this plugin makes webpack not only looking for package.json, but also for a bower.json with a main-field
        new webpack.ResolverPlugin([
            new webpack.ResolverPlugin.DirectoryDescriptionFilePlugin("bower.json", ["main"])
        ]),
        new webpack.DefinePlugin({
          VERSION: JSON.stringify('alpha'),
          DEVEL: true
        })
    ],

    module: {
        loaders: [
            // Exports Angular
            { test: /[\/]angular\.js$/, loader: "exports?angular" },
            // Script Loaders
//            { test: /\.coffee$/, loader: "coffee" },
            { test: /\.coffee$/, loader: "coffee-loader" },

            { test: /\.html$/, loader: "html" },
//            { test: /\.jade$/, loader: "jade" },
            { test: /\.jade$/, loader: "template-html-loader" },
            // Style Loaders, style! inlines the css into the bundle files
            { test: /\.css$/, loader: "style!css" },
            { test: /\.scss$/, loader: "style!css!sass" },
            { test: /\.less$/, loader: "style!css!less" },
            { test: /\.styl$/, loader: "style!css!stylus" }
        ]
    }

};

I've tried a variety of things and this is what I have at the moment.  So, I'm missing something small and obvious right?

@dschissler
Hi,
After searching on Google and tracing through threads, I ended up here.  Did you ever figure out a solution for your original post?  After many configurations, I'm thinking webpack doesn't allow this....
Thanks!

@phnguyen89 Yes I found a solution a long time ago and you can see it in the gulpfile.webpack.js of my project.  It will likely involve the context, resolve.root, resolve.modulesDirectories and resolve.resolveLoader.

So this can be closed? I think it's a rather specific issue and not related to the less-loader.

@dschissler Thank you for the prompt reply.  Unfortunately, I wasn't able to achieve a purely webpack.config.js file even with those settings.  I gave up and just added the additional ".babelrc" file (sigh):
var path = require('path');
var nodeModulesRoot = path.resolve(__dirname, 'node_modules');
var distRoot = path.join(__dirname, 'dist');

module.exports = {
  context: path.resolve('..'),
  entry: [
    './client/index.jsx'
  ],
  module: {
    loaders: [
      {
        exclude: /node_modules/,
        loader: 'babel-loader',
        query: {
          plugins: ['transform-runtime'],
          presets: ['react', 'es2015']
        },
        test: /\.jsx?$/
      }
    ]
  },
  resolve: {
    extensions: ['', '.js', '.jsx'],
    modulesDirectories: [
      'node_modules'
    ],
    root: [
      nodeModulesRoot
    ]
  },
  resolveLoader: {
    root: nodeModulesRoot
  },
  output: {
    path: distRoot,
    publicPath: '/',
    filename: 'bundle.js'
  },
  devServer: {
    contentBase: distRoot
  },
};

@jhnns Man, you guys are fast.  I only got to replying just now.  I only wanted to ask a question, not raise an issue.  Thanks for closing it!

@phnguyen89 Last time that I tried about two weeks ago I wasn't able to get Babel 6 working with my setup so I've dropped back to version 5 for now.  I'm going to give it a bit of additional time and then I will raise issues about it.  It will be involved and difficult to figure out what is happening so I'm putting it off as long as possible.

@dschissler I'm glad that others are having issue :)  ".babelrc" is actually more portable than writing it in webpack lingo, so it's all how you look at the problem haha.  The only way I was able to make it work in webpack.config.js is to use absolute paths in the module.loaders's query.plugins and query.presets.  I suppose that's fine, but feels quite redundant.

@phnguyen89 I just upgraded to postcss 5 and I had to re-specify in the plugin which paths to look inside.


Fix the Contact Us button in the MVP so it collapses into the single column in the mobile view

@ntknguyen one minor request, can you add a little bit of spacing after the button (margin-bottom-1) so its not right on top of the text when you are on a phone size screen?

@ntknguyen one minor request, can you add a little bit of spacing after the button (margin-bottom-1) so its not right on top of the text when you are on a phone size screen?

It's fixed.

@ntknguyen looks great. Thanks!


rip.properties?

Cant seem to find rip.properties, and keep getting error null trying to rip a tumblr page. Im using windows 10.

For the record: it's auto-generated on start.

it wasnt generated but after i kept changing settings it appeared

If the file is not found it should use the version of the file that is packaged into the JAR. If you can get any additional information about the problem, that would be appreciated.


Setter equivalent of X509_REQ_get0_signature for OpenSSL > 1.1.0 appears missing

I've got a chunk of code that carefully builds up a clean certificate from another request.
   X509_REQ * req == read input
....
   creq = X509_REQ_new();
   ...

and at some point I need to populate the signature alg. Which I used to do with:
  creq->sig_alg = X509_ALGOR_dup(req->sig_alg);
I understand that this const struct X509_algor_st * has become private around 1.1.x.
Now I can get the value with help of the getter; and duplicate it fine:

   const X509_ALGOR *psigalg;

   X509_REQ_get0_signature(req, NULL /* no need for signature */,&psigalg);

   X509_ALGOR *pcsigalg = X509_ALGOR_dup((X509_ALGOR *) psigalg);

but what is the right way to fill copy this into the X509_REQ struct ? Is there a setter I've missed? Or what is the modern 1.1.1 way of `not poking' in the DER struct during an d2i/i2d dance
Or should one approach this very differently ?
PS: actual code @ line 900 in https://source.redwax.eu/projects/RS/repos/mod_csr/browse/mod_csr.c#900

I guess - that all is needed is below -- but not sure how to deal with the libcrypto.num details.
Dw.
`diff --git a/crypto/x509/x509_req.c b/crypto/x509/x509_req.c
index 7b88dbcd2..a05ac8bc6 100644
--- a/crypto/x509/x509_req.c
+++ b/crypto/x509/x509_req.c
@@ -285,6 +285,14 @@ void X509_REQ_get0_signature(const X509_REQ *req, const ASN1_BIT_STRING **psig,
if (palg != NULL)
*palg = &req->sig_alg;
}
+void X509_REQ_set0_signature(X509_REQ *req, ASN1_BIT_STRING *psig,

                        X509_ALGOR *palg)

+{

if (psig != NULL)

   req->signature = psig;

if (palg != NULL)

   req->sig_alg = palg;

+}
int X509_REQ_get_signature_nid(const X509_REQ *req)
{
diff --git a/doc/crypto/X509_get0_signature.pod b/doc/crypto/X509_get0_signature.pod
index 61a2dda98..db68e86cd 100644
--- a/doc/crypto/X509_get0_signature.pod
+++ b/doc/crypto/X509_get0_signature.pod
@@ -2,7 +2,7 @@
=head1 NAME
-X509_get0_signature, X509_get_signature_nid, X509_get0_tbs_sigalg,
+X509_get0_signature, X509_REQ_set0_signature, X509_get_signature_nid, X509_get0_tbs_sigalg,
X509_REQ_get0_signature, X509_REQ_get_signature_nid, X509_CRL_get0_signature,
X509_CRL_get_signature_nid - signature information
@@ -13,6 +13,8 @@ X509_CRL_get_signature_nid - signature information
void X509_get0_signature(const ASN1_BIT_STRING **psig,
const X509_ALGOR **palg,
const X509 *x);

void X509_REQ_set0_signature(X509_REQ *req, ASN1_BIT_STRING *psigOrNull,

                        X509_ALGOR *palgOrNull)

int X509_get_signature_nid(const X509 *x);
const X509_ALGOR *X509_get0_tbs_sigalg(const X509 *x);

@@ -32,6 +34,9 @@ X509_get0_signature() sets B<*psig> to the signature of B and B<*palg>
to the signature algorithm of B. The values returned are internal
pointers which B be freed up after the call.
+X509_get0_signature() is the equivalent setter. If NULL is passed - that
+value is ignored; and the value in *req is not overwritten.
+
X509_get0_tbs_sigalg() returns the signature algorithm in the signed
portion of B.
diff --git a/include/openssl/x509.h b/include/openssl/x509.h
index 038cef953..2c9e4663a 100644
--- a/include/openssl/x509.h
+++ b/include/openssl/x509.h
@@ -660,6 +660,8 @@ X509_NAME *X509_REQ_get_subject_name(const X509_REQ *req);
int X509_REQ_set_subject_name(X509_REQ *req, X509_NAME *name);
void X509_REQ_get0_signature(const X509_REQ *req, const ASN1_BIT_STRING **psig,
const X509_ALGOR **palg);
+void X509_REQ_set0_signature(X509_REQ *req, ASN1_BIT_STRING *psigOrNull,

                        X509_ALGOR *palgOrNull);

int X509_REQ_get_signature_nid(const X509_REQ *req);
int i2d_re_X509_REQ_tbs(X509_REQ *req, unsigned char **pp);
int X509_REQ_set_pubkey(X509_REQ *x, EVP_PKEY *pkey);
diff --git a/util/libcrypto.num b/util/libcrypto.num
index 21d46d27b..99e7e591f 100644
--- a/util/libcrypto.num
+++ b/util/libcrypto.num
@@ -4208,3 +4208,4 @@ OCSP_RESPID_set_by_key                  4158	1_1_0a	EXIST::FUNCTION:OCSP
OCSP_RESPID_match                       4159	1_1_0a	EXIST::FUNCTION:OCSP
ASN1_ITEM_lookup                        4160	1_1_1	EXIST::FUNCTION:
ASN1_ITEM_get                           4161	1_1_1	EXIST::FUNCTION:
+X509_REQ_set0_signature			4161	1_1_1	EXIST::FUNCTION:
`

Again - as a file.
patch-set0_signature.txt

Could you please create a PR on github?
On Tue, Dec 3, 2019 at 12:41 AM dirkx-saltrino-2019 <
<EMAIL_ADDRESS>wrote:

Again - as a file.
patch-set0_signature.txt
https://github.com/openssl/openssl/files/3913411/patch-set0_signature.txt
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/openssl/openssl/issues/10560?email_source=notifications&email_token=ADJFVE6HF2RMDBQ6B56DVZTQWV6J5A5CNFSM4JUGPIU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFVIO2I#issuecomment-560629609,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADJFVE3C5LGNWIQJMEANO4LQWV6J5ANCNFSM4JUGPIUQ
.

--
SY, Dmitry Belyavsky

not sure how to deal with the libcrypto.num details.

Instead of adding this line manually...
+X509_REQ_set0_signature			4161	1_1_1	EXIST::FUNCTION:

... just run make update, this adds the missing entry to util/libcrypto.num.

Closed  -replaced by a broader PR


Support for Tags in cloudflare_logpush_job

Current Terraform and Cloudflare provider version
terraform version: v0.15.1
provider terraform-provider-cloudflare_v2.20.0
Description
When using an AWS S3 bucket in combination with Cloudflare's cloudflare_logpush_job, setting a lifecycle policy on the AWS S3 bucket leads to deletion of logpush_ownership_challenge file, as the logs must be in the same folder as the ownership challenge file (as stated in #1019 and #1024).
Currently, when setting the AWS S3 Bucket Lifecycle Policy to automatically delete items older than N days, it is only possible to filter items based on a Prefix (i.e., a subdirectory) or Tags.
Since the destination_conf attribute of the logpush_ownership_challenge resource must be equal to destination_conf attribute of the logpush_job (as also stated in #1019 and #1024), the challenge file is affected by the lifecycle-rule and will be deleted when the retention period is met.
Alternative Fix: remove the requirement that the destination_conf attribute of the logpush_ownership_challenge resource must be equal to the destination_conf attribute of the logpush_job.
Components:

aws_s3_bucket
logpush_job
logpush_ownership_challenge

Use cases
By supporting tags, it is possible to adjust the lifecycle filter for AWS S3 buckets (by only deleting log files with a given tag), and to prevent the deletion of the challenge file.
Potential Terraform configuration
resource "cloudflare_logpush_job" "example_job" {
  enabled = true
  zone_id = "d41d8cd98f00b204e9800998ecf8427e"
  name = "My-logpush-job"
  logpull_options = "fields=RayID,ClientIP,EdgeStartTimestamp&timestamps=rfc3339"
  destination_conf = "s3://my-bucket-path?region=us-west-2"
  ownership_challenge = "0000000000000"
  dataset = "http_requests"
  tags = {"key": "value",  "name": "log-file", "retention_period": "7days"}
}

References
#1019
#954

This isn't the place for service requests, just the implantation of managing resources.
I'd recommend dropping your use use into Cloudflare Support or your account representative for them to chat to the service team.

@dmuse89 Did you find a solution/workaround for this? I am facing the same issue. Right now I am recreating the ownership challenge on every Terraform run. That creates unnecessary noise, so I am not super happy with it.

Thanks for your reply. I ended up solving this a bit simpler than I expected, so I leave this here for the next person wondering how to do this.
I hard coded the value of the ownership challenge. This is documented as "Example Usage (manual inspection of S3 bucket)" on the docs: https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/logpush_job

What I ended up doing was to place the log files inside a prefix in the destination bucket, place my lifecycle policy in this prefix only and then, by using the s3_object_copy, copied the ownership file outside the directory/prefix.
Then, with a aws_s3_object data resource read the copied file like this
data "aws_s3_object" "challenge_copy_file" {
  bucket = "my_tests"
  key    = "challenge_copy.txt"
  depends_on = [ aws_s3_object_copy.challenge_copy ] # this is needed because TF will try to read this data block before applying anything and in result fail
}

and used it in the Cloudflare resources.
The good thing is that the s3_object_copy only checks if the destination exists, so the source file can be deleted without any issue.


deps<EMAIL_ADDRESS>

Mostly updates fastclick to a newer, more maintained library.
Also should ensure things work with iOS 9.3 Safari.

Closing for now, will reopen if there's a need for this later.