Method render_template does not use blueprint specified template_folder

File structure:

project -
      - templates \
                - a \
                      search.html
                - b \
                      search.html
      - start.py
      - myapplication \
                - test.py

Start file:

# start.py
from flask import Flask
from myapplication.test import test_blueprint

app = Flask(__name__)
app.register_blueprint(test_blueprint)

my application

# test.py
from flask import Blueprint, render_template, abort
from jinja2 import TemplateNotFound

test_blueprint = Blueprint('test_blueprint', __name__,
                        template_folder='absolute_path_to_project/templates/a')
                        # YES, I specified the absolute path to the template a

@test_blueprint.route('/test')
def show():
    try:
        return render_template(''search.html") # HERE is the problem
    except TemplateNotFound:
        abort(404)

Is this a problem?
# Actually, I want to render a/search.html
# But, render_template() does not use test_blueprint's template_folder
# render_template() search the template list, find the first search.html, then render
# So, maybe render a/search.html or b/search.html
# This depend on the sequence of the template list

What is the output, and what did you expect?

The render_template function should look like this:
def render_template(template_name_or_list, **context):
    """Renders a template from the template folder with the given
    context.
    :param template_name_or_list: the name of the template to be
                                  rendered, or an iterable with template names
                                  the first one existing will be rendered
    :param context: the variables that should be available in the
                    context of the template.
    """
    ctx = _app_ctx_stack.top
    ctx.app.update_template_context(context)

    if request.blueprint is not None:
        blueprint_template_folder = current_app.blueprints[request.blueprint]
        if (blueprint_template_folder is not None and
            isinstance(template_name_or_list, string_types)):
            template = ctx.app.jinja_env.get_or_select_template(
                os.path.join(blueprint_template_folder, template_name_or_list))

    if template is None:
        template = ctx.app.jinja_env.get_or_select_template(template_name_or_list)

    return _render(template, context, ctx.app)

Alternatively we could modify the Jinja2 context so get_or_select_template looks in the blueprint's template folder first.

I encountered the same bug. This is a huge bug that could brake a lot of blueprints.
I build up a simple example to reproduce the bug.  You can look at the comments inside code for more details.
https://github.com/ael-code/flask_bugs

I suspect that I'm running into this as well.  Is this just the way it works?
Maybe it would help matters if the stock Blueprint example was updated to show the intended way to run multiple blueprints.  The existing example doesn't seem to address what we are asking about in this issue.  I think it could be updated to show two different Blueprints, running side by side, with their own copy of /templates.  They both should have an index.html showing how to load one or the other from the Blueprint's-specified templates folder.  That way, we will understand clearly what is the intended way to do this.
I'd do it myself, except I don't know if what I'm writing below is the intended way or if there actually is a bug here.
It feels like:

flask.render_template ought to be searching the blueprint's template_folder first and then fall back to app.template_folder (and deal with the subsequent problem of caching internally)
or maybe the Blueprint class should have a render_template works like that (search the blueprint's template_folder first and then fall back to app.template_folder.

For my project, I am currently doing things as follows.  This is working, but it's messy because everything in mixed together in the /templates folder.
/myapp
  /blueprint1
    views.py
  /blueprint2
    views.py
  /templates
    /blueprint1
      index.html
    /blueprint2
      index.html

blueprint1/views.py
bp = Blueprint('blueprint1', __name__, static_folder='static', template_folder='templates")
@bp.route("/")
def index():
	return flask.render_template('blueprint1/index.html')

blueprint2/views.py
bp = Blueprint('blueprint2', __name__, static_folder='static', template_folder='templates")
@bp.route("/")
def index():
	return flask.render_template('blueprint2/index.html')

See how we have to specify the name of the blueprint path in render_template?
I was hoping for this to work instead:
blueprint1/views.py
bp = Blueprint('blueprint1', __name__, static_folder='static', template_folder='blueprint1/templates")
@bp.route("/")
def index():
	return flask.render_template('index.html')

blueprint2/views.py
bp = Blueprint('blueprint2', __name__, static_folder='static', template_folder='blueprint2/templates")
@bp.route("/")
def index():
	return flask.render_template('index.html')

An improvement is to add a render_template() function to the Blueprint object that is guaranteed to search the template_folder first and fall back to app.template_folder if it doesn't exist.
bp = Blueprint('blueprint2', __name__, static_folder='static', template_folder='blueprint2/templates")
@bp.route("/")
def index():
	return bp.render_template('index.html')

I was running into the same problem, I have changed the code supplied by @alanhamlett to work with the way paths were supposed to be supplied to blueprints.
def render_template(template_name_or_list, **context):
    """
    Renders a template from the template folder with the given
    context.
    :param template_name_or_list: the name of the template to be
                                  rendered, or an iterable with template names
                                  the first one existing will be rendered
    :param context: the variables that should be available in the
                    context of the template.
    """
    ctx = _app_ctx_stack.top
    ctx.app.update_template_context(context)

    template = None

    if request.blueprint is not None and \
            isinstance(template_name_or_list, string_types):
        bp = current_app.blueprints[request.blueprint]
        try:
            template = bp.jinja_loader.load(ctx.app.jinja_env,
                                            template_name_or_list,
                                            ctx.app.jinja_env.globals)
        except TemplateNotFound:
            pass

    if template is None:
        template = ctx.app.jinja_env\
            .get_or_select_template(template_name_or_list)

    return _render(template, context, ctx.app)

I think it is really weird and unwanted behaviour that one blueprint searches in another blueprint's template_folder. This still does not fix that, but it at least searches is own folder first, then the app folder and then all the other blueprints their folders.

The point of searching the app template folder first is so that an app developer can override templates supplied by a blueprint from an extension.  Is this still possible with these changes?

No, this method first searches the blueprint template folder. Overriding of the blueprint templates would require some changes that I'm not sure how to do. The reason for this is that currently all the blueprint template folders are added to the global app search path. This means that loading from the app will try to load from any blueprint as well, which means another blueprint folder can be searched before the blueprint of the route that is actually used.
The above code fixes this by first searching the blueprint template folder and only after nothing is found it will search all other options. To fix this something would need to change in the app loader, which requires more work. Since I do not need overriding of templates I did not implement this. Feel free to change the code though.

Preferring the blueprint template folder over the application's template folder would break the existing contract between app's and blueprints. This contract is used to much success across the extension ecosystem. For example, the Flask-Security extension mounts a blueprint to an application and comes packaged with a set of minimal default templates. The app developer can then override the stock templates by placing their own files in the app's template folder under the same path as the extension's blueprint.

I know my fix is not sufficient for inclussion, but for my use case this was sufficient. That any blueprint searches in any other blueprints template folder is totally crazy. It makes no sense and I can see no reason for it, especially since the search order seems to be "random".

I second the sentiment.  It's weird how they are searching one another's folders.  The order makes no sense.  I can't control it. I keep going in circles.

Are there any extensions depending on this feature? It seems this should be a config variable for the extension, instead of implicitly overriding templates.

Still encountering this issue but across two different blueprints, one of my templates is being rendered instead of the other one when they have the same name.

I'm super surprised this is an issue. Is the expectation that template loading is blueprint relative?

I actually think that a lot would be won if Flask would warn if someone renders a template without a path separator.

At least for small applications it makes lots of sense to just render index.html
Anyway, having e.g. 'foo/*.html' for templates from the foo blueprint is pretty useful, but it adds a lot of redundancy if you end up with a folder structure like this: yourapp/modules/foo/templates/foo/bar.html (i.e. you already have a folder for the blueprint and end up repeating it inside the templates folder)

@mitsuhiko
You mean when it doesn't start with a path seperator?
Because I don't think I've ever seen it called with a starting slash.
http://flask.pocoo.org/docs/dev/quickstart/#rendering-templates

I guess he meant template paths specifying only a file, nothing else

Also I think it would at least be very nice if it was configurable per blueprint if it template loading is relative. If you try to separate different parts of big applications using blueprints it would be very useful.
However, if you use them for extensions where templates could be overridden by a user you don't want it.

The main problem is that you cannot make it fully separate without introducing some way to reference a template from the main app or another blueprint, e.g. in a {% extends %} or {% import %}

What's the point of having the template_folder argument for Blueprint if it doesn't do anything?

@alanhamlett what do you mean it does not do anything? It adds the folder to the load path.

@JelteF i mean that if we render from a blueprint we could just warn if someone does render_template('foo.html') instead of render_template('blueprint_name/foo.html'). (eg: no '/' in the path).

How would you avoid the warning if you don't want any url rules in your main app (e.g. because you have an api and frontend blueprint)? Having templates/frontend/index.html and using render_template('frontend/index.html')? Feels like something where at config option would be appropriate to disable the warning for applications where there's only one blueprint that contains templates and adding a subdirectory would just add extra noise.

@ThiefMaster
I think the search order I proposed under my code sample would work there as well.

Search template directory of this blueprint
Search main app template directory
Search all other template directories

Furthermore, a big issue I have with how it works at the moment is that the load order is non deterministic. If the template name is not in the main app template directory it will choose the first one it finds in a blueprint directory based on the key order in a dictionary. This changes on every startup. So debugging this issue is also very hard.

@JelteF this will never happen for a few reasons:

it will cause templates to go by different names which causes caching issues internally among many other problems (extends breaking hilariously)
we did something like this with modules before and the general feedback was that it's a terrible idea and very confusing
it would break many patterns that people currently have

That said we could have a conversation about render_template('./foo.html') being relative to the current blueprint's name or something. Eg: it expands to render_template('blueprint_name/foo.html').

@mitsuhiko
What I think @alanhamlett means is that template_folder sounds like it should at least try to search earlier in that folder than in other template_folders
Maybe a different search order would make more sense then for backwards compatibility:

Search main app template directory.
Search directory of this blueprint
Search directories of other blueprints (in a deterministic order).

The ./foo.html syntax would be fine for me as well.

@JelteF "Maybe a different search order would make more sense then for backwards compatibility:"
That would only encourage that you could load templates by non qualified names. Because templates are cached by their name however this would mean that whatever blueprint loads first would cache the template for the other.
So this just cannot work. The only thing that could be done is to allow a template name to be prefixed with the blueprint name. However this would still require you to store the template as templates/blueprint-name/index.html.

Honestly: changing this behavior is opening an enormous can of worms. I rather make debugging this easier.

@mitsuhiko
Storing the template like that is exactly what I'm doing right now. It's just that I don't want to manually prefix my template names with blueprint-name/ every time. Which is also why I would be totally fine with the ./ syntax or another keyword argument such as template_prefix.

Because templates are cached by their name however this would mean that whatever blueprint loads first would cache the template for the other.

When is this cache used? I've been using the render_template code I posted above for almost a year now and have not noticed any of such problems.

@JelteF the templates are cached in app.jinja_env.cache.

Aahh, so it's only for extends, import and stuff like that. I think I just use the fully qualified names there.

It's also used for successive calls to render_template.

Ok, looking at the code in: https://github.com/pallets/flask/issues/1361#issuecomment-121737708
It seems I only use the cache when it's not in the blueprint directory. That explains it. In that case that "solution" is indeed even less nice than I thought.
In that case I think the ./ solution together with a warning when using no slashes would be very nice. Replacing the dot with the template name would be fine as a default and maybe a keyword argument should be added to allow it to change it to something different.

@JelteF yeah. Your chance is entirely bypassing the load cache which means that you are recompiling the template on every request which is ridiculously slow.

I'll try and make a pull request for that when I have time.

@JelteF no need. Such a change will not be merged.

I will close this as wontfix and someone can open up a new issue on the improved diagnostics if they are not good enough.

Wait, why would the ./ solution not be merged?

@jeltef thanks, that's exactly what I meant just too tired to have said it correctly. Telling a blueprint to use a template folder should mean to check that folder first.

I think we're conflating two related issues.

The app takes precedence over the blueprint when loading a template.  This is good behavior, it allows the app to customize the templates that extensions provide.
If the app does not override the template, the templates are still merged together, so it's undefined which template will get loaded if two blueprints use the same directory structure.

The first is not going to change.  The second is what this issue was actually opened about.  I'm not sure what the fix is here besides ensuring that blueprints don't use the same directories in their template folders.

Given that blueprints are iterated over in the order they were registered, I think the correct resolution here is to more thoroughly document the lookup order somewhere in the docs.  We've already added the EXPLAIN_TEMPLATE_LOADING config, which can also help show what is going on.

@davidism we just want to prevent repeating ourselves by specifying the template_folder very time our blueprints call render_template. How about an optional argument to Blueprint.__init__ that says to load templates from the blueprint first?

class MyBlueprint(Blueprint):
    def render_template(self, name, **kwargs):
        return render_template('/'.join((self.name, name)), **kwargs)

bp = MyBlueprint('blog', __name__)

@bp.route('/')
def index():
    return bp.render_template('index.html')

The actual template path is still blog/index.html, so it can still be overridden.  But you don't have to type blog/.

Can we have this added as a Snippet?
Overwriting the include template block for this blueprint would allow relative paths there too.

Blueprint "template_folder" only works if I put the absolute path...
I would like to use: template_folder='templates/my_sub_folder'
and: render_template('file.html')
not: render_template('my_sub_folder/file.html')
Maybe there is a bug in "template_folder".

@JohnRobson read the thread before posting in it please.  This has been explained.  There is a snippet that simplifies your request two posts up.

@davidism, you created that workaround because "template_folder" doesn't work, right?

It does work, exactly as expected.  The template folder is relative to the blueprint's location and blueprints have lower priority than the app and previously registered blueprints.  This is not a bug.

Ok, thank you for your explanation. I expect to use it to setup the templates folder for my blueprint file.
So, if 'template_folder' works properly, how can I use it? eg: bp = Blueprint('bp', name, template_folder='templates/bpsubfolder')

Your expectation is not how template lookups work.  You create a subdirectory in order to distinguish the blueprint templates.  You pass a template folder to a blueprint in order to append it to the search path, not to just use those templates.  So don't pass your subdirectory as the template folder, pass it during render template or use the subclass I wrote above.
my_project/
    my_app/
        __init__.py
        users/
            __init__.py
            templates/
                users/
                    index.html

bp = Blueprint('users', __name__, template_folder='templates')

@bp.route('/')
def index():
    return render_template('name/index.html')

I'm already passing the sub-folder during render_template, thank you.

Attached is a small patch for a better seach path of render_template() (modifies def iter_loaders(...)). Like @dagostinelli suggested, it attempts to load the template first from the current blueprint, then the app and then the other blueprints.
It doesn't really solve the problem of refering to a specific template file with duplicate name from a blueprint to another blueprint ... Here I would suggest another method (render_template) that has an additional argument for the blueprint name / app. It can almost be like in url_for() in that it refers to an endpoint.
templating.py.patch.txt

This suffers the same problems as the other "fixes": it breaks the ability to override templates provided by extensions.

@davidism with @Querela's patch we just need a new attribute on Blueprint called templates_prefer_local that will only load from the local Blueprint first when True. The default could be False so all existing implementations would not change.

@alanhamlett that would probably be better.
@davidism for routes from the blueprint I would normally expect to have a higher priority for its own templates and then the app and others. I'm not too sure on how it interfers with extensions.
Additionally I tried to extend the render_template method. It now has an additional arguement for a name/list of names of Blueprints in which the template should be searched only. This patch is on top of my previous one. Maybe in this way it would be better. Here Blueprints could be addressed something like in url_for but not as endpoints. Well, I mean:
# assuming that mod_a und mod_b are blueprints with their own template
# directories (both are named index.html)

@mod_a.route('/mod_a')
def func_a():
    # to only search for index.html in blueprint mod_b
    return render_template('index.html', 'mod_b', title='Hello World', ...)

@mod_a.route('/mod_a2')
def func_a2():
    # normal behaviour
    return render_template('index.html', title='Hello World', ...)

templating.patch.new_render_template.txt

For anyone who stumbles upon this as I did many months ago, and are as perfectly alright violating the blueprint contract as they are understanding of it's implications, I ended up writing a tiny Flask plugin to solve this issue.

Is this issue resolved. I am facing same issue. I switched from Django to flask. Now looking at shortcoming thinking of moving back to Django.

Our stance is that there is no issue here. Blueprint template folders do not take precedence over the app template folder. This is to allow an app to override extension templates. If you are in a situation where you want to go the other direction, please reconsider exactly why you want to do that.
Locking this as there is nothing more to say on the issue, it just keeps going in circles.


RoutePattern does not throw on same duplicate default values

@rynowak I'm merging this because both this and IEndpointSelector need a feature branch, and IEndpointSelector has breaking changes for MVC that need to be fixed before they break the build.

Will deal with any comments you have in a follow up.


[release-1.5] feat(bootstrap): add ability to declare stat regexes

This is an automated cherry-pick of #21119

Gentle cc @fpesce

@istio-testing: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

pilot-e2e-envoyv2-v1alpha3_istio_release-1.5
547304b508bfe85660843330914601d048391e55
link
/test pilot-e2e-envoyv2-v1alpha3_istio_release-1.5

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test pilot-e2e-envoyv2-v1alpha3_istio_release-1.5


Add delay in seconds to refresh function

@ismaail Can you kindly explain why there should be a delay? We used to have one with the old methodology we used, but it never worked right. What benefits does having a delay provide?

For me the problem is with .php files, specially when using virtual server (Vagrant)
livereload is faster than web server to flush php cache (I think).

OK. I can see that. But adding a setTimeout is not the way to solve this. Chokidar supports intervals. If you want to investigate that (https://github.com/paulmillr/chokidar) I can integrate it in. But I'm closing this implementation.


Any docs?

any docs ? any instalation ?
i see you have .env config is compatible with laravel ? any help ?

Hi!
Well, this project was created for own requirements, but quite configurable. If you can share more details of your need with me, maybe I can assist you with your problem?
Best regards,
Bostjan


Teams + SSO: ensure to send an oauth card with sign in link + change HTTP failure response from 409 to 412

@carlosscastro I'm not clear on the timing for this work, so please merge when you're ready.


flutter create should make a gradle-based project for Android

When you run flutter create, you should get an Android project that uses gradle.  Most developers will eventually need to use gradle to integrate with third-party SDKs and to add their own Java code.  We should start them off in that mode so that the transition is smooth.

@jakobr-google is doing exactly this as we speak.  I suspect we have a dupe of this somewhere.

Duplicate of #4796


RFP - ShareMouse

ShareMouse is a software that let you share your mouse and keyboard with multiple networked computers. It is similar to Synergy in this package, but I think ShareMouse is more user friendly and stable.

I have uploaded this package here.


On https://prod-preview.openshift.io - creating new space - creating project fails

Steps to recreate:

User Logins in
Selects “Sign in” with GitHub
Create Space
Select Scenario Driven Planning
Get going dialog - selects "technology stack and pipeline"

A dialog with this error message is opened:
Something went wrong while attempting to perform this operation ...
The [ forge-quick-start :: begin :: 0 ] command failed or only partially succeeded ...
name => AppGenerator Api Error
message =>
===============================================================================
Http Error
Http Status Code :
Http Status :
===============================================================================
Http Error Body:
isTrusted => true
===============================================================================

Unlinking the account resolved this error:
http://sso.prod-preview.openshift.io/auth/realms/fabric8/account/identity


Winged hapuku camera and power

wasn;'t working
camera wasn't plugged in, and USB cable was crappy

solved it


find_all error in kg download

I got error while running the download command.
kg download -u '******' -p '******' --competition 'https://www.kaggle.com/c/data-science-bowl-2017'
I got the following error:

Starting new HTTPS connection (1): www.kaggle.com
'NoneType' object has no attribute 'find_all'

Any suggestions??

@nitish11 to me accepting ToS on kaggle website helped with the same error

After accepting ToS, the below command is the accepted solution.
kg download -u '******' -p '******' --competition 'data-science-bowl-2017'

Still getting this error (with 'dogs-vs-cats' competition, after accepting TOS), and it looks to be due to line 27 of download.py – it's looking for an element with an id that doesn't exist, then trying to find 'a' elements within that.
Looks like kaggle switched to react and doesn't display all download links anymore – could this have something to do with it or am I missing something?

Hey msgrasser - this has been thwarting me as well..  your comments made me think.. I wonder if I reinstall kg.. when I tried, I found that pip was out of date..  'pip install --upgrade pip' solved that.. followed by 'pip install --upgrade kaggle-cli'.  Now when I run kg download.. it works!  Hope this helps.

Super helpful, calliope7, will try that – thank you very much!

Haha, so it looks like in the time between my comment and your comment there was another issue + subsequent PR that addressed this.
pip being out of date wasn't actually the cause - the cause was in fact what I had hypothesized regarding the dynamic content generation for kaggle's web app. Regardless, when you upgraded kaggle-cli you must've caught that latest update made in the PR mentioned above.
Thanks for the tip, and I'm super glad to see how active this repo is!

Yup that looks like it.. Thanks for the followup.

Hey guys getting-'Response' object has no attribute 'soup' error for all competitions.Any thoughts on how to go about it?

Guys it is solved I needed to put '-' between all words in the competition name and all will be small case


Office 365 is not found on the first page of the document

🛑 IMPORTANT: You can get your feedback addressed faster if you use the comment section for the article in which you encountered a problem.
It is not a problem but I believe many people will search for Office 365 and it is not there. It may be under Office but I bet many people will put a Ctrl+F to search for "Office 365".
Link to article:
Tell us where exactly a problem occurs.
https://docs.microsoft.com/en-us/
Problem:
What's wrong with the article?
There is no Office 365 on the first page.

@adkinn Can you take a look at this feedback?

@Powerhelmsman I believe it's represented by the "Office" and "Microsoft 365" links. You could confirm with Joe Mazzotta to see if this is correct.

Office 365 link will shortly be added to the Docs directory page on Docs home page. PR was created and reviewed/approved today. Pending merge and push occuring shortly: https://docs.microsoft.com/en-us/


How to custom text FloatActionsMenu?

Anyone knows how to change the font of the text using a custom font?
Thanks!

Even I am searching for same, I think they need to add this functionality.


Add an example and type enhancement for TextStreamer

Added a GenerationConfig option for TextStreamer and included an example in the documentation as part of #394. I used the example you provided directly and tried to adhere to the repository's style as much as possible, but please let me know if anything deviates from the standard.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

If you don’t mind, I can contribute a chat-streaming example to the /example folder. Would it be okay to use Vite with Tailwind styling like others (or possibly Shadcn and Next.js), or is there a standard dependency stack for this repository examples?

That would be much appreciated! However, could you instead contribute it to https://github.com/huggingface/transformers.js-examples? We're in the process of moving all examples to that repository.
In terms of stack, feel free to use what you like, but it should contribute to the example you are trying to showcase. For example, for a simple streaming example, I would recommend keeping it as simple as possible (just to highlight the important part; i.e., streaming). That said, it could be useful to see how to pass this information back from a web worker too, so feel free to contribute two examples if you'd like the second one to be more "fully-featured". In that case, feel free to use what you feel most comfortable with.
One other thing to note is that Next.js introduces another level of complexity since it can be run as a non-static site. Anyway, things to consider - I look forward to the contribution! Thanks!


[nrf noup] fs: littlefs: Add littlefs to pm.yml to support the littlefs backend

Add #ifdef for littlefs so that littlefs can be used in the same way
as NFFS with pm.yml configurations.
Signed-off-by: Sigvart Hovland<EMAIL_ADDRESS>

Could you verify the change by running some settings test on pca10090ns with this backend? (If supported)

Fails on some of the littlefs tests with nrf9160 . I'll try with nrf52840 which boats as white-listed board I think some configuration is wrong for the test.

History in this repository has been rewritten. Please rebase this PR.

This repository's history has been rewritten for the v1.2 release. Please rebase this PR if still relevant, or close it if no longer needed. Thanks!

DNM until Upmerge #296 will be merged

More than a year old now, does it still apply?
Needs to be rebased after 1.4.0 history rewrite.

Not sure if it's still needed.

The majority of this code should be in the nrf repo now anyway. Also the flash_map.h changes should make this obsolete. Closing.


DXVK 2.5.2 and Warhammer ROR

hello  the game was running perfectly in DXVK 2.5.1 the moment I upgraded to DXVK 2.5.2 the graphic was mixed up and in some areas the texture turned to all black, switching back to DXVK 2.5.1 fixed the issue no graphic bug
thanks for your work

Probably already fixed. Try the latest master.

@K0bin Maybe tag v2.5.3 since this flood of tickets will keep growing. 😅

We'll probably do that pretty soon in after New Years Eve.


Dependabot Setup

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
Dependabot Setup
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?

NONE

/ok-to-test

/close

/reopen

/test

@BenTheElder thank you so much!

not sure why "/test all" didn't work (see: https://prow.k8s.io/command-help#test) but I guess explicitly specifying all of them works ...

/retest

/kind cleanup


Update suggested supported browsers in readme

for .babelrc, currently it looks like this:
                "targets": {
                    "chrome": "63",
                    "safari": "12",
                    "ios": "12",
                    "edge": "18",
                    "ie": "11"
                }

Assigning Abi as an expert, otherwise I'll do it myself later...
Or anyone else is welcome to help

@darkwebdev  We should use https://github.com/eBay/browserslist-config which is the official and maintained regularly by the browser policy team. Also, it is used by Marko (see https://github.com/eBay/ebayui-core/blob/84ba5f3ddb47105d4d3a0baba71941c181c6ef25/package.json#L39-L41) so we should use the same for consistency.
Babel will use browserslist automatically.

I've updated manually for now, I hope babel can use browser list from package.json...


build: remove ts-node usages

We no longer need ts-node at runtime like we did before (this was either a cli thing or a config thing), we can remove ts-node from most packages.
That leaves only a couple that actually used it:

store for codegen, but this was easy to swap with tsx (used elsewhere)
noise for test:hardhat but the command wasn't working or in use anyway, so I just reduced it to what it should be assuming hardhat adds TS support

we could also just migrate the test cases over to foundry

Helps a bit with #612

gonna merge this since this is mostly clean up and want this for templates PR that I am starting on next


Automatic CT orientation from DICOM

Apparently the DICOM standard encodes patient orientation

https://dicom.nema.org/MEDICAL/DICOM/current/output/chtml/part03/sect_C.7.6.2.html
https://dicom.innolitics.com/ciods/ct-image/image-plane/00200037

Can try to build an automatic orientation parser towards automated X-ray/CT registration.

Also can use direction matrices stored in DICOM / NIFTI formats to orient the CT in physical space: http://itksnap.org/pmwiki/pmwiki.php?n=Documentation.DirectionMatrices

Convert DICOM to NIFTI, then get affine matrix for orientation: https://github.com/rordenlab/dcm2niix


Improve sample data

Shout out to @jkleinsc for getting us a way to load sample data for development, and creating the initial sample data file. :metal: :zap:
The sample data could use some improvement, though. Currently most screens only have 1 example, and there are still a few that don't have any from the sample data. I'm happy to use the current sample data as the basis and work on making it more extensive.
Would be good to document the process for exporting a data dump, too.

@jglovier I'd like to unassign this and put this out in Help Wanted. I feel like someone who's not you could work on this.

@tangollama

cc @jkleinsc for issue number reference.

@jglovier can we close this or no?

@tangollama I still think there is big room for improvement here worth leaving this open. Right now the sample data file we load only has a small set of data (many views have 1 or 0 records), which isn't really vert effective for working with the app in a truly representative way.
I've been able to get around that via the private anonymized data set we have, but I'd like to keep this open until we have a set of anonymized sample data that any contributor can use when getting setup. Correct me if I'm mistaken on whether that data is still private or available to all contributors.

@jglovier What would this case entail to do? As in would I just create plausible sounding records arbitrarily, or does it need to be based off of this anonymized sample data?

@Eric-Arellano I think the anonymized sample data is the best route here, and actually @jkleinsc has something like this in progress IIRC, which is why we don't have a help-wanted label on this issue. @jkleinsc @tangollama can you confirm?

@jglovier @Eric-Arellano yes I have a dataset, I just haven't had time to add it.
@jglovier the dataset is about 39M over 6 files, so I'm hesitant to add that to the repo.

@jglovier the dataset is about 39M over 6 files, so I'm hesitant to add that to the repo.

Sure, that's fair. We could workaround that by just adding instructions on how to get it and install it to the README, and then host the dataset elsewhere like a dropbox account or something.

Since it's CouchDB, is it possible to set up a public database that we all can sync from?


Issue with ESRI Vector Tiles

Hi,
Not sure if this is OL issue or OLMS, but here is what I noticed:
ESRI vector tiles basemaps worked fine in OL 6.x and its corresponding OLMS. The newer version sort of works, but throws an error (the map can still function).
I am using EPSG:4326

issue one: probably ESRI error as their TileServer incorrectly (??) describes its tiles. It uses EPSG:3857 structure for rows/columns (1x1, 4x4, 16x16, ...) vs (1x2, 2x4, 4x8, ...). So in order to prevent OL using their definition, I replaced the TileServer reference to nothing, otherwise OL fails to work; unless I am missing a step:

fetch('https://www.arcgis.com/sharing/rest/content/items/cc8072363d614d96b08b2361a09979a5/resources/styles/root.json?f=pjson')
    .then(response => {
        return response.json();
    })
    .then (data => {
        data.sources.esri.url = '';
        applyStyle(layer, data, "", undefined, tileGrid.getResolutions());

    })

The map loads but keeps throwing following error:
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'map')  at getTileJson (util.js:163:31)  at apply.js:525:5  at new Promise (<anonymous>)  at setupVectorSource (apply.js:524:10)  at assignSource (apply.js:226:20)  at onChange (apply.js:310:15)  at apply.js:368:15

To Reproduce
Here is my code:
import MVT from 'ol/format/MVT.js';
import Map from 'ol/Map.js';
import VectorTileLayer from 'ol/layer/VectorTile';
import VectorTileSource from 'ol/source/VectorTile';
import View from 'ol/View.js';
import { applyStyle } from 'ol-mapbox-style';
import { get as projGet } from "ol/proj";
import {createXYZ} from 'ol/tilegrid';

function tileUrlFunction(tileCoord) {
  return (
    'https://basemaps.arcgis.com/arcgis/rest/services/World_Basemap_GCS_v2/VectorTileServer/tile/{z}/{y}/{x}.pbf'
  )
    .replace('{z}', String(tileCoord[0] + 1))
    .replace('{x}', String(tileCoord[1]))
    .replace('{y}', String(tileCoord[2]));
}

    const tileGrid = createXYZ({
        extent: [-180, -90, 180, 90],
        tileSize: 512,
        maxResolution: 180 / 512,
        maxZoom: 15,
    });

  const layer = new VectorTileLayer({
    declutter: true,
    source: new VectorTileSource({
        format: new MVT(),
        projection: 'EPSG:4326',
        tileGrid: tileGrid,
        tileUrlFunction: tileUrlFunction,
    }),
  });

    const map = new Map({
    layers: [
        layer
    ],
    target: 'map',
    view: new View({
        center: [0, 0],
        minZoom: 0,
        zoom: 0,
        projection: projGet("EPSG:4326"),
    }),
    });

    fetch('https://www.arcgis.com/sharing/rest/content/items/cc8072363d614d96b08b2361a09979a5/resources/styles/root.json?f=pjson')
    .then(response => {
        return response.json();
    })
    .then (data => {
        data.sources.esri.url = '';
        applyStyle(layer, data, "", undefined, tileGrid.getResolutions());

    })
    .catch(error => {
        console.error("Error processing.");
    });

Expected behavior

either have the ability to ignore the default ESRI settings
or not throw Error

Perhaps there is another way of loading ESRI vector tile layers?
Anyway, thanks in advance and great job on maintaining this awesome library!
Otmar

OpenLayers only supports the standard EPSG:3857 Mapbox tilejson specification https://github.com/mapbox/tilejson-spec
For vector tiles in other projections see https://openlayers.org/en/latest/examples/vector-tiles-4326.html for a hardcoded tile grid solution.
ESRI uses a service json for custom tile grids and projections.  Another option is to fetch and parse that to create a tile grid in a custom projection as in https://github.com/OrdnanceSurvey/OS-Data-Hub-API-Demos/blob/master/OSVectorTileAPI/OpenLayers/map.js

Regarding issue 1, that's indeed a problem with the way Esri references the tiles in the TileJSON. They are aware of the problem, but haven't fixed it yet. However, you can work around it using the transformRequest option, as shown in the esri-transformrequest example:
https://github.com/openlayers/ol-mapbox-style/blob/336c77b6a83f97423affa17e250ce8640f99bdc1/examples/esri-transformrequest.js#L8-L14
And issue 2 probably comes from the way you're removing the tilejson from the source, so it will solve itself when you apply the solution for issue 1 correctly. Your modified code could look like this:
import 'ol/ol.css';
import VectorTileLayer from 'ol/layer/VectorTile.js';
import VectorTileSource from 'ol/source/VectorTile.js';
import {Map, View} from 'ol/index.js';
import {applyBackground, applyStyle} from 'ol-mapbox-style';
import {createXYZ} from 'ol/tilegrid.js';

const url =
  'https://www.arcgis.com/sharing/rest/content/items/cc8072363d614d96b08b2361a09979a5/resources/styles/root.json?f=pjson';

// Match the server resolutions
const tileGrid = createXYZ({
  extent: [-180, -90, 180, 90],
  tileSize: 512,
});

const layer = new VectorTileLayer({
  declutter: true,
  source: new VectorTileSource({
    projection: 'EPSG:4326',
    tileGrid: tileGrid,
  }),
});
applyStyle(layer, url, {
  resolutions: tileGrid.getResolutions(),
  transformRequest(url, type) {
    if (type === 'Source') {
      return new Request(
        url.replace('/VectorTileServer', '/VectorTileServer/')
      );
    }
  },
});
applyBackground(layer, url);

new Map({
  target: 'map',
  layers: [layer],
  view: new View({
    projection: 'EPSG:4326',
    zoom: 0,
    center: [0, 30],
  }),
});

The above is basically the https://openlayers.org/en/latest/examples/vector-tiles-4326.html example with your style json and the transformRequest option.

As in https://github.com/openlayers/ol-mapbox-style/issues/603 it will also work without a transformRequest by using https://basemaps.arcgis.com/arcgis/rest/services/World_Basemap_GCS_v2/VectorTileServer/resources/styles/ as the style url
https://codesandbox.io/s/vector-tiles-4326-forked-dfbxpw?file=/main.js

Thanks a lot, that did the trick!

I just published v9.5.0, which adds a new projection option. With that option, your above example can be simplified to https://github.com/openlayers/ol-mapbox-style/blob/v9.5.0/examples/esri-4326.js (see it live on<EMAIL_ADDRESS>

Does it also need a maxResolution option to determine the tile grid?  ESRI uses a single tile at level 0 while in https://openlayers.org/en/latest/examples/vector-tiles-4326.html MapTiler uses 2 tiles.

For Esri, the above setup works. For MapTiler it does not. The rationale behind the addition of the projection option to ol-mapbox-style was the assumption of a standard tile grid for each projection, which is defined by zoom level 0 filling a single tile at the projection's validity extent as obtained from the EPSG database. For other setups, more manual work is necessary.

I would like to add, that while the simplification works, it only loads the default ESRI country styling. I had to combine both of your original answers to get it working. Try different urls to see the difference.
//        let url = 'https://www.arcgis.com/sharing/rest/content/items/cc8072363d614d96b08b2361a09979a5/resources/styles/root.json?f=pjson';
//        let url = 'https://www.arcgis.com/sharing/rest/content/items/55253142ea534123882314f0d880ddab/resources/styles/root.json?f=pjson';
        let url = 'https://www.arcgis.com/sharing/rest/content/items/670c2a9ee3b949a8991b99df425abd0d/resources/styles/root.json?f=pjson';
        const tileGrid = createXYZ({
            extent: [-180, -90, 180, 90],
            tileSize: 512,
            maxResolution: 180 / 256,
            maxZoom: 22,
        });

        let layer = new VectorTileLayer({
            declutter: true,
            source: new VectorTileSource({
                projection: 'EPSG:4326',
                tileGrid: tileGrid,
            }),
        });

        applyStyle(layer, url, {
            resolutions: tileGrid.getResolutions(),
            transformRequest(url, type) {
            if (type === 'Source') {
                return new Request(
                url.replace('/VectorTileServer', '/VectorTileServer/')
                );
            }
            },
        });
        applyBackground(layer, url);

Also, if anyone else is interested in finding additional ESRI 4326 basemaps:
https://www.arcgis.com/home/group.html?id=702026e41f6641fb85da88efe79dc166&view=list&categories=["%2FCategories%2FGCS+WGS84"]#content

@vuratek Your code can be simplified to
import 'ol/ol.css';
import {apply} from 'ol-mapbox-style';

apply(
  'map',
  'https://www.arcgis.com/sharing/rest/content/items/670c2a9ee3b949a8991b99df425abd0d/resources/styles/root.json?f=pjson',
  {
    projection: 'EPSG:4326',
    transformRequest(url, type) {
      if (type === 'Source') {
        return new Request(
          url.replace('/VectorTileServer', '/VectorTileServer/')
        );
      }
    }
  }
);


Add doxygen support

Run doxygen in project root dir with doxygen doxygen.txt

Very cool.  Thanks!


Partial _custom.scss is not in file .gitignore

Is there a special reason why the partial _custom.scss was not added to .gitignore ?
https://github.com/twbs/bootstrap/blob/v4-dev/scss/_custom.scss
https://github.com/twbs/bootstrap/blob/v4-dev/.gitignore

It's imported into the main Sass file, so without it there to start you'd have compilation errors. https://github.com/twbs/bootstrap/blob/v4-dev/scss/bootstrap.scss

Now if you make changes to _custom.scss. There is a conflict.
.gitignore doesn't remove files.

But now if I make changes to _custom.scss there is a conflict with the next git pull

This is only if we make changes to the file on the same lines that you've changed.

Thanks for coming back on this.
That sounds like removing the comment from _custom.scss (I replaced it with my own) also could cause the conflict.
Later today I will reset the file to original.
And copy+paste back my customised variables after the comment. To see what happens.

@BobbyBabes, if _custom.scss isn't working for you, you can create your own SCSS file in your project and use @import "bootstrap";

@bardiharborow That's what I was already doing (second paragraph in my opening comment).
@mdo is right though. Making a new clone and modifying scss/_custom.scss works flawlessly with git pull.
Although I still get the conflict error (my local changes would be overwritten by git pull) in the first test clone of Bootstrap that I had made. Checking out the original scss/_custom.scss file is the only way (I know of) to get rid of the error.


Fit into vertical screens

Hi!
It would be nice if the UI took up more space when on a vertical (mobile) screen

Decker is designed for landscape aspect ratios. You may get better results, though, if you check "Decker -> Fullscreen" in web-decker. This will cause most browsers to hide their UI and stretch Decker to fill the available space.
For example, here's how Decker looks on a Fire 8 tablet in fullscreen mode:

That doesn’t seem to do anything on iOS :(

Did some research and a few experiments, and apparently the phone version of Safari for iOS has no support for fullscreen APIs. May work on an iPad, but I can't really do anything for iPhones.


How could I use my own XSD to verify the XML(stix)？

I need to use the particular XSD to verify the XML, but I can not change the XSD to replace the TAXII_XMLMessageBinding_Schema_11.xsd ? T_T How to do Q_Q?

hey @p0561199. unfortunately it is not something you can easily extend.
opentaxii uses schema validators from libtaxii project to validate TAXII XML. It does unpack embedded XML, like STIX (unfortunately) but this is more of a side effect. No validation of content is supported yet

@p0561199 If this is still an issue, I might give this a shot using the stix-validator. Question:
Do you want this on a per-collection basis or on a per-instance basis (only files going to a certain collection should be validated vs all files ingested by that instance of opentaxii)?

We have recently made a decision to consider this project feature-complete. It means we still maintain it, however we focus only on bug fixes. As this issue is a feature request, unfortunately we won’t add it and therefore I’m closing this issue.


[FLINK-26532][metrics][test] using the numRecordsSend counter to get the correct metric

What is the purpose of the change
Fix the flaky test in SinkMetricsITCase. The previous check solution could not provide the guaranty of getting the correct metrics.
Since the target metric numRecordsSend for this test case is initialized in SinkWriterMetricGroup via parent OperatorMetricGroup and the SinkMetricsITCase is working on the abstract layer of OperatorMetricGroup, there is no direct access to the SinkWriterMetricGroup and to the numRecordsSend counter.
Brief change log
(for example:)

check the metric by getting the numRecordsSend metric from InMemoryReporter and cast it to Counter

Verifying this change
This change is to fix a flaky test.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)

I do not fully understand the fix. AFAIK the InternalSinkWriterMetricGroup delegates everything to the parent metric group so when updating something in the SinkWriterMetricGroup it should be immediately available also in the parent group.

It is true. But just because the metric is initialized via the parent, does not means it can be accessed by the parent metric group. if you know any other way to do it except adding extra @VisibleForTesting method into AbstractMetricGroup which will break it current design, please let me know.


[WIP] Add API server PKI tool

This patch adds a tool intended to simplify the creation of the TLS
certificates required by the API server. For details see the README.md
inside the patch.
This patch addresses issue #474.

@fabiand, @stu-gott Please review.
I consider this work in progress, but I think that it is complete enough for you to review.
I didn't test it with the real API server, but I did test it with the sample API server, and it seems to work correctly.

@jhernand this is crazy :) I'll need to deep dive into it, but nice to see this contribution.
@deads2k could you take a look as well?

@jhernand I'm excited to see this land. I'm reviewing your PR now. Thanks for the contribution!

For the specific use case of the virt-apiserver, it also needs access to the requestheader ca cert. That doesn't need to be stored as a secret per-se, but I had added it there as a convenient place to send that piece of data to the virt-apiserver. Is that something we'd want to continue to store in the same secret?

Thanks for the review @stu-gott !
Yes, the CA certificate is part of the "PKI stuff", so I think it should also be part of the secret, even if it isn't conceptually a secret. Note that the certificate isn't conceptually a a secret either. But it makes sense (and is easier) to have all the PKI stuff in the same place.
Regarding the size of the main function, yes, I will try to make it smaller.

The new version of the pull request just fixes some typos, and a problem that made the build fail in Travis. Hopefully it will now build correctly. But none of the important comments has been addressed. Will do that in the next version of the pull request.

After testing this deeper, I realized that the API aggregator doesn't accept certificates signed by the cluster CA by default. That is surprising. It means that the APIService resource always needs to be created explicitly specifying a CA bundle, otherwise the aggregator won't be able to talk to that service, even if has a certificate signed by the CA of the cluster. I think that requires a change in the approach of this tool: it would be better to have this tool do the complete registration of the API service. That way it can create the APIService resource with the correct attributes. I am working in that directoy, so please don't review yet.

The new patch uses the approach mentioned in the previous comment: it creates the secret, the service and the API registration. That way it can configure all of them with matching CA certificates.
In addition I have simplified the main function, to make it smaller.
I verified that this works correctly with the sample API server, setting the configuration so that it does verify TLS certificates: insecureSkipTLSVerify: false.
I think it is ready for another round of reviews now.

btw, I forgot to say this in my initial review. I this tool is awesome :+1:  Great work so far @jhernand

@davidvossel thanks for the review. All the comments make sense, I will try to address them. My initial impression is that the thing most complicated to implement can be the locking, and I am not sure if it is really worth, as the chances of having two API services with the same API group and version registering simultaneously is low. I will start working first in the other changes that you suggest.
By the way, regarding locking, does Kubernetes provide any locking mechanism which can be used for this purpose?

The new version of the pull request contains the following changes, as requested during review:

The tool is now idempotent: it only does the changes that it needs to do. For example, it will not create a new private key, or a new CSR, if one already exists (probably from a previous execution).

The tool has a new -auto-approve option to indicate if it should try to approve the CSR itself. The default value is false.

The tool has a new -approve-timeout option to indicate how long to wait for the approval of the CSR. The default value is zero, which means that it will wait for ever.

The tool uses now the KubeVirt logging package instead of glog.

The only review comment that isn't addressed is locking. I still think that it isn't worth. If you think that is needed I can try to implement locking based on mounting a shared persistent volume, and use a file in that volume as the lock, assuming that all the pods that use the tool will use the same persistent volume.

The only review comment that isn't addressed is locking. I still think that it isn't worth. If you think that is needed I can try to implement locking based on mounting a shared persistent volume, and use a file in that volume as the lock, assuming that all the pods that use the tool will use the same persistent volume.

@jhernand
Here's why I thought locking would be important. Let me know what you think. It's possible locking is a non-issue if this tool isn't going to be deployed the way I'm assuming it is.
If we launch an aggregated api server in a replica set, then we'll have multiple api server pods initializing around the exact same time.  If this tool is an init container for that replicated pod, then that also means we'll have multiple instances of this tool running in parallel when the api servers are coming online for the first time.  Without locking, my concern is multiple instances of this tool could end up stomping on each other.
For locking, I think you could just reuse the leader election locking. There's an example in pkg/virt-controller/watch/aplication.go. No PVC would be necessary.

virt-apiserver is a cluster level resource, so will be a deployment not a
daemonset.

virt-apiserver is a cluster level resource, so will be a deployment not a
daemonset.

Right, the deployment will look similar to virt-controller's, where we have at least 2 instances of the api server running at any time.
The difference between virt-controller and the api servers is that the api servers work in active/active mode. There will be a load balancer and all replicated instances will need to be capable of managing requests.

Does the pki tool need to be an initialier - or could it be a Job which is run once when all manifests get deployed?

good point. We still need to work through some of that.
I'm okay with punting on locking for now then until we're sure we need it. I'll give this update another detailed review soon.

The PKI tool does not need to be an initialization container, it could run from other place. Before or after the other manifests are deployed, that isn't really important. If it runs like this then locking won't be relevant, which would simplify it. But where would it run from? What would deploy the manifests? How would that thing call this tool?

By the way, I always thoght that we should have some kind of "deploy tool", to do all those things.

Hey, I just discovered that the k8s certificate generation code is in client-go. Most of the code here can potentially be replaced by it: https://github.com/kubernetes/client-go/tree/master/util/cert

Here an example how the kubelet does tls bootstraping based on it: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/certificate/bootstrap/bootstrap.go
Another interesting resource: https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/

Most of the code here can potentially be replaced by it: https://github.com/kubernetes/client-go/tree/master/util/cert

Good find :+1:
This gives us code we can reuse to generate certs, but doesn't necessarily help with the approval process or persisting any of this info in the k8s secret though.

This gives us code we can reuse to generate certs, but doesn't necessarily help with the approval process or persisting any of this info in the k8s secret though.

It is already there :) https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/certificates/approver/sarapprove.go. The service account issuing it needs permissions which look like this:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: auto-approve-csrs-for-group
subjects:
- kind: User
  name: system:serviceaccount:kube-system:kubevirt-infra
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: approve-node-client-csr
  apiGroup: rbac.authorization.k8s.io

And the certificate needs to be carefully drafted right now: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/certificates/approver/sarapprove.go#L179

this tool is still necessary for service creation and secret management.


[Usage]: VLLM serve Gemma 2 9B it with more than 4096 tokens

Your current environment
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.5 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35
Python version: 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-101-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 535.161.07
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             6
On-line CPU(s) list:                0-5
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7443 24-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 3
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           5699.59
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip pku ospke vaes vpclmulqdq rdpid arch_capabilities
Virtualization:                     AMD-V
L1d cache:                          192 KiB (3 instances)
L1i cache:                          192 KiB (3 instances)
L2 cache:                           1.5 MiB (3 instances)
L3 cache:                           16 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-5
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Mitigation; safe RET, no microcode
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected
Versions of relevant libraries:
[pip3] flashinfer==0.1.6+cu121torch2.4
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==<IP_ADDRESS>
[pip3] nvidia-cuda-cupti-cu12==12.1.105
[pip3] nvidia-cuda-nvrtc-cu12==12.1.105
[pip3] nvidia-cuda-runtime-cu12==12.1.105
[pip3] nvidia-cudnn-cu12==<IP_ADDRESS>
[pip3] nvidia-cufft-cu12==<IP_ADDRESS>
[pip3] nvidia-curand-cu12==<IP_ADDRESS>
[pip3] nvidia-cusolver-cu12==<IP_ADDRESS>
[pip3] nvidia-cusparse-cu12==<IP_ADDRESS>
[pip3] nvidia-ml-py==12.560.30
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] nvidia-nvjitlink-cu12==12.6.68
[pip3] nvidia-nvtx-cu12==12.1.105
[pip3] pyzmq==26.2.0
[pip3] torch==2.4.0
[pip3] torchvision==0.19.0
[pip3] transformers==4.44.2
[pip3] triton==3.0.0
[conda] No relevant packages
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.6.1.post2@9ba0817ff1eb514f51cc6de9cb8e16c98d6ee44f
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	0-5	0		N/A
Legend:
X    = Self
SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX  = Connection traversing at most a single PCIe bridge
NV#  = Connection traversing a bonded set of # NVLinks
How would you like to use vllm
I want to run inference of google/gemma-2-9b-it with the 8192 tokens supported by the model, using vllm serve. Originally the system responds that it can only run with 4096 tokens. I tried with:
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
export VLLM_ATTENTION_BACKEND=FLASHINFER
vllm serve google/gemma-2-9b-it --enforce-eager --port=8080 --gpu-memory-utilization=0.95 --dtype=auto --disable-sliding-window --max-model-len=8192 --trust-remote-code
also tried with:
export VLLM_ATTENTION_BACKEND=FLASH_ATTN
If I then try sending it more than 4096 tokens the server either responds with garbage or stops.
Has anyone been able to run Gemma 2 successfully and stable with 8192 tokens? How? Thanks
Before submitting a new issue...

[x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Already considered in issues 6220 and 8580

@agorodetzky still there is no action on this?


Reuse embedded OSGi container in tests

This speeds up build time from 17m to 11m on my machine by reusing OSGi framework containers where possible.  That also fixes an OSGi container which is causing lots of build failures on Jenkins.
See the last commit which is the only thing new, and the OsgiManager class is the main significant difference in that.  Other things are just wiring to control bundle reuse or not.
This includes #645 and #647 so if this build passes we can assume the failures in those PR's are ignorable (they seem to be due to memory leaks).  But merge those before merging this (and I can rebase this if needed once they are merged.)
/cc @neykov @aledsage @geomacy

This all looks good to me; tests all pass, with speedup (just for brooklyn-server build) on my machine from 12:20 min to 09:05 min.
Will merge.


fixed: Advanced hierarchical Table of Contents

#590

Fixing some test scripts.

@MartijnR
Thanks for the response.
- a
> group one
    - b
    - c
- d
- **e**
> group three
    - **f**
    - **g**
    > **group three-two**
        - **h**
        - **i**

However, I still not understand why items in ** will be shown in ToC, because if we navigate form with buttons (Next and Back buttons), those items will not shown as its own page.

Hi @gushil, The way I understood it ToC content and structure has basically nothing to do with pages. But I'm going to check with ODK Collect, because I might be misunderstanding!

Yes, this is correct. I understand the confusion though. Just having the ToC code as part of pages.js is confusing. This was actually the case for historical reasons (since the current ToC does relate to pages). I think it is worth considering creating toc.js maybe, and importing that in either form.js (best?) or pages.js. Please consider that, and use the same toc-update trigger as pages.js uses to update pages.

Thanks for the suggestions!
Will look into that possibility and implementing it next.

Almost there. Need to refactor and do some styling.


unset xcodeProjectDir throws exception: Ambiguous method overloading

For the following line, when xcodeProjectDir == null, it could match either File(File, String) or File(String, String), so it throws an exception as it's ambiguous. Even though xcodeProjectDir is explicitly type to String.
    @OutputFile
    File getPodFile() { new File(project.j2objcConfig.xcodeProjectDir, "Podfile") }

Here's the exception:
groovy.lang.GroovyRuntimeException: Ambiguous method overloading for method java.io.File#<init>.
Cannot resolve which method to invoke for [null, class java.lang.String] due to overlapping prototypes between:
	[class java.lang.String, class java.lang.String]
	[class java.io.File, class java.lang.String]
	at com.github.j2objccontrib.j2objcgradle.tasks.J2objcXcodeTask.getPodFile(J2objcXcodeTask.groovy:53)

groovyness:
doesn't matter that xcodeProjectDir is of type String.  We are using Groovy in its (default) dynamic typing scheme, so method resolution is strictly at runtime.  At runtime, null can have any non-primitive type.
we should probably just have an explicit validation step before that that also throws a meaningful message like if using the xcode task, you must set that config flag.

PR #227 has that validation step when the action is called


Vote for Operator Name

We are building an operator that installs, updates and deletes application runtimes on Kubernetes nodes. This means we are not limited to WebAssembly runtimes, as was the case with its predecessor KWasm. We could install any runtime environment such as Kata, https://bun.sh/ or similar.
The original idea references the Operator as containerd-shim-lifecylce-operator. While we are focusing on contained shims for now, there is a possible demand for alternative container runtimes (such as CRI-O) and no technical reason not to support those.
With this issue I'd like to collect ideas and eventually vote on a proposal. Once we have settled on a name, we will make sure to rename the project as well as identifiers in the code accordingly.
Ideas:

Kubernetes Runtime Lifecycle Manager
RuntimeClass Manager

Both are fine, but I have a small preference for "RuntimeClass Manager"

For me, the main task is not to manage the RuntimeClass, but to set up the runtime. The RuntimeClass is just the tip of the iceberg, but since the user will mainly interact with the RuntimeClass, I can agree with this name. To stay in the picture, it is the visible part of the iceberg. 😉


A problem of Multiclass Classification

As far as I understtod, your code supports only a binary classification problem. I could not find any information in the paper regarding the classes (the "Words"/"Subjects" are the classes?). I am trying to use this for a multi-class problem. Since pairing has been done for frame sequences of each video (9 of them)  with the corresponding speech spectrogram and MFEC features, I suppose there will be no problem if one changes the number of classes.
When I change number of classes from 2 to 6, I get this error, can you help me?
Epoch 1, Minibatch 1 of 15 , Minibatch Loss= 1056.706787, EER= 0.50000, AUC= 0.33333, AP= 0.69683, contrib = 8 pairs
Epoch 1, Minibatch 2 of 15 , Minibatch Loss= 1793.572998, EER= 0.50000, AUC= 0.55000, AP= 0.61167, contrib = 9 pairs
Epoch 1, Minibatch 3 of 15 , Minibatch Loss= 1273.130249, EER= 0.50000, AUC= 0.62500, AP= 0.80417, contrib = 6 pairs
Epoch 1, Minibatch 4 of 15 , Minibatch Loss= 1280.513916, EER= 0.25000, AUC= 0.60714, AP= 0.81829, contrib = 11 pairs
Epoch 1, Minibatch 5 of 15 , Minibatch Loss= 1651.882568, EER= 0.40000, AUC= 0.60000, AP= 0.67778, contrib = 9 pairs
Epoch 1, Minibatch 6 of 15 , Minibatch Loss= 1395.890381, EER= 0.40000, AUC= 0.48000, AP= 0.53429, contrib = 10 pairs
Epoch 1, Minibatch 7 of 15 , Minibatch Loss= 1423.493164, EER= 0.27273, AUC= 0.63636, AP= 0.58000, contrib = 16 pairs
Epoch 1, Minibatch 8 of 15 , Minibatch Loss= 1248.631836, EER= 0.50000, AUC= 0.55000, AP= 0.61167, contrib = 9 pairs
Epoch 1, Minibatch 9 of 15 , Minibatch Loss= 1377.684937, EER= 0.50000, AUC= 0.54167, AP= 0.74385, contrib = 10 pairs
Epoch 1, Minibatch 10 of 15 , Minibatch Loss= 1460.154419, EER= 0.33333, AUC= 0.83333, AP= 0.88750, contrib = 7 pairs
Epoch 1, Minibatch 11 of 15 , Minibatch Loss= 1794.762451, EER= 0.40000, AUC= 0.33333, AP= 0.67771, contrib = 17 pairs
Epoch 1, Minibatch 12 of 15 , Minibatch Loss= 1140.301392, EER= 0.50000, AUC= 0.37500, AP= 0.36667, contrib = 6 pairs
Epoch 1, Minibatch 13 of 15 , Minibatch Loss= 1273.781738, EER= 0.66667, AUC= 0.47619, AP= 0.51664, contrib = 16 pairs
Epoch 1, Minibatch 14 of 15 , Minibatch Loss= 989.276489, EER= 0.50000, AUC= 0.58333, AP= 0.36667, contrib = 8 pairs
Epoch 1, Minibatch 15 of 15 , Minibatch Loss= 1625.663696, EER= 0.33333, AUC= 0.83333, AP= 0.95028, contrib = 11 pairs
TESTING: Epoch 1, Minibatch 1 of 3
TESTING: Epoch 1, Minibatch 2 of 3
Traceback (most recent call last):
  File "/media/Data/Scripts/lip-reading-deeplearning/code/training_evaluation/train.py", line 667, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "/media/Data/Scripts/lip-reading-deeplearning/code/training_evaluation/train.py", line 659, in main
    score_dissimilarity_vector[i * batch_k_validation:(i + 1) * batch_k_validation])
  File "/media/Data/Scripts/lip-reading-deeplearning/code/training_evaluation/roc_curve/calculate_roc.py", line 16, in calculate_eer_auc_ap
    AUC = metrics.roc_auc_score(label, -distance, average='macro', sample_weight=None)
  File "/usr/local/lib/python2.7/dist-packages/sklearn/metrics/ranking.py", line 277, in roc_auc_score
    sample_weight=sample_weight)
  File "/usr/local/lib/python2.7/dist-packages/sklearn/metrics/base.py", line 72, in _average_binary_score
TESTING: Epoch 1, Minibatch 3 of 3
    raise ValueError("{0} format is not supported".format(y_type))
ValueError: multiclass format is not supported

Is this because of ROC calculation (line 659 of train.py) for multi-class classification? Can you tell me which parts need modification, maybe I missed something.

The paper is for verification. It only deals with binary classification as a consequence.
I do not understand what you meant about class definition. The point is when you create pairs for comparison, it is a binary classification problem. No matter who is saying what. So the subject and the spoken words do not matter.
Please read the associated paper for further details.

@asinatorfi I understand that this is a verification paper for binary classification. I already changed the metrics and my problem is solved. Sorry, I forgot to close the issue. Thanks a lot for your helps.

My pleasure. Glad to see things are solved.


Issue #177 Allow multiple TLD and resolver configs

This PR allows landrush.tld to take an array of TLD values. Landrush will create OS X resolver configurations for each TLD. There is no change to the existing syntax for assigning a single value.
An example Vagrant configuration would look like
config.landrush.tld = ['vm.example.com', 'vm.test.org']
config.landrush.host 'www.vm.example.com', HOST_IP
config.landrush.host 'www.vm.test.org', HOST_IP

An example benefit for multiple TLD comes when running multiple websites on a single virtual machine using Apache/Nginx name-based vhosting. That is, a VM can host www.vm.example.com and www.vm.test.org and clients can resolve both domains.

I pushed a rebased version of this pull request to here - https://github.com/hferentschik/landrush/tree/issue-177

On a tangent, any idea why the automatic host configuration is only applied for OS X? Obviously due to /etc/resolver its a bit easier, but one could also run the required dnsmasq commands for Linux, right? Same for Windows really. There I am working on a prototype, but it is work in progress.

auto config for Windows and Linux is outside my expertise but certainly would be a welcomed feature.

any ideas when this may be merged?

any ideas when this may be merged?

I was a bit on a back-burner recently. I am also still wondering whether the suggested approach is the way to go. Finally got around finishing my pull request for Windows network configuration. Looking at this shortly again...

I was a bit on a back-burner recently. I am also still wondering whether the suggested approach is the way to go. Finally got around finishing my pull request for Windows network configuration. Looking at this shortly again...

Awesome super excited about it

Hi ! Any ideas when this may be merged ?

@hferentschik ... any ideas on when this is will make it in :-D ? Still a very useful feature? Do you need any help merging and publishing the GEM? Or is lack of linux support an issue here?
/cc: @mheiges

It's been more than two years, so I just want to ask if this project is still alive and if this PR is likely to ever be merged?

Superseded by pull request #332 which merges this approach onto master.


'BUILTIN_LED' was not declared in this scope

May I know which pin we should define for BUILTIN_LED?

sorry, i realized i forgot to change to ESP board at the Arduino Board Manager setting.
please close the issue. thanks.


Bug: Functional properties make inference non-working in case of circular type parameter constraints like T extends M

Bug Report
🔎 Search Terms
Circular type parameter constraint, inference change due to presence of functional property
Related: #40439
🕗 Version & Regression Information
Tested with v4.3.4
⏯ Playground Link
Playground
💻 Code
declare const m: <T extends M<T>>(m: T) => T

type M<Self, V = Prop<Self, "v">> =
  { v: V
  , t?: (v: V) => void
  }

type Prop<T, K> = K extends keyof T ? T[K] : "error"

// expected : M<{ v: number, t?: (v: number) => void }>
// actual   : M<unknown>
m({
  v: 1,
  t: k => {}
})

// expected : M<{ v: number }>
// actual   : M<{ v: number }>
m({
  v: 1
})

🙁 Actual behavior
In first function call of m, the parameter is inferred as M<unknown> in other words the TypeScript couldn't infer anything.
🙂 Expected behavior
In first function call of m, the parameter should be inferred as M<{ v: number, t?: (v: number) => void }>. Especially given that removal of t works as seen in the second function call. Presence of a functional property shouldn't make the inference completely non-working.

Oh shoot I realized it's same as #40439 ahaha


Typos in platform_impl.nim

unkown -> unknown
I have a PR ready, but I'm not sure should I edit the version number

Please open the PR without editing the version number.


mocha-typescript-watch regular expressions pass-through

Mocha allows regular expressions to match test cases and you can decide which tests to skip. Would be nice if the mocha-typescript-watch would allow to pass-through these parameters back to mocha.
  -g, --grep <pattern>                    only run tests matching <pattern>
  -f, --fgrep <string>                    only run tests containing <string>

You can register any arguments you would like to pass down to mocha here:
https://github.com/PanayotCankov/mocha-typescript/blob/master/bin/watch.ts#L37
And pass them to the mocha process here:
https://github.com/PanayotCankov/mocha-typescript/blob/master/bin/watch.ts#L71
Or alternatively we can think of a way to provide arbitrary arguments to mocha using something like mocha-typescript-watch -- <everything-passes-down-to-mocha>, but I will have to research what is supported in yargs.

You mean with something like  -m ./node_modules/mocha/bin/_mocha -g expression ?

@truhlikfredy, exactly, :)
This has been resolved within the 1.0.19 patch of the modules.
You can now execute:
$ ./node_modules/.bin/mocha-typescript-watch -g <regex pattern>

Or should you have the package.json script:
  "scripts": {
    "test": "tsc && mocha",
    "watch": "mocha-typescript-watch"
  },

You can pass the -f or -g flags through npm run like:
$ npm run watch -- -g <regex pattern>

As a side note I never use these flags, usually once I start the watcher, when I focus on a suite or test I add the @test @only test() {} decorator or change from describe() or it() to describe.only() or it.only(), and after a save, the watcher will compile and execute only the marked tests. I find it much more straight forward than having to copy the name to the command line and re-run the mocha script.
On the other hand the grep and fgrep are useful in case you want to run groups of tests scattered in various suites, so here it is :)


typo: private key, not public key

the deterministic nonce is generated by hashing the private [not public] key and the hash of the message.

Oh, I see this is a duplicate fix from #104 - the fix got reverted in #117.

ACK. Thank you. Deterministic nonce is necessarily a private scalar.

Thanks @MaxHillebrand !
I think this would actually be clearer if it said "use sha256(bytes) on the private key and message" (since the preimage should be (bytes(d) || m))


error::Cannot read property 'startsWith' of undefined

G'day:
I am testing this action locally with act like so:
act -j deployToFirebase -v
My stack:

macOS Catalina
node v14.3.0
yarn 1.22.5
act 0.2.14

I am running into a problem, and here's the partial stack trace:
[Build and Deploy to Firebase/deployToFirebase] Exec command '[node<EMAIL_ADDRESS>[Build and Deploy to Firebase/deployToFirebase]   ❗  ::error::Cannot read property 'startsWith' of undefined
[Build and Deploy to Firebase/deployToFirebase]   ❌  Failure - dorny/paths-filter@v2
...
DEBU[0040] exit with `FAILURE`: 1
DEBU[0040] exit with `FAILURE`: 1
DEBU[0040] exit with `FAILURE`: 1
Error: exit with `FAILURE`: 1

My workflow file deployToFirebase.yml looks like:
name: Build and Deploy to Firebase

on:
  push:
    branches: [ develop ]

jobs:
  deployToFirebase:

    runs-on: ubuntu-latest
    env:
      CI: ""

    steps:
    - name: Check out code
      uses: actions/checkout@master
    - uses: dorny/paths-filter@v2
      id: filter
      with:
        filters: |
          frontend:
            - 'src/**'
          backend:
            - 'functions/**'

     # run only if 'frontend' (hosting) files were changed
    - name: frontend steps
      if: steps.filter.outputs.frontend == 'true'
      run: |
        echo "<<<<<<<<<<<<<<<<<<<<<<< frontend changed"
     # run only if 'backend' (functions) files were changed
    - name: functions steps
      if: steps.filter.outputs.backend == 'true'
      run: |
        echo ">>>>>>>>>>>>>>>>>>>>>>> functions changed"

What am I not doing right? Or is there anything special I have to do in order to use this action?
I am also open to suggestions on how else to test locally to save on the hassle of first sending the code to github and then waiting for the workflow result

Hello, thanks for reporting the issue.
Looks like act is not providing all context data this action needs.
Most probably ref or repository.default_branch from event payload are missing.
There are no checks for it as those fields should be mandatory.
I will add some default behavior for this case and rest should work.
Should be fixed in few days.

Hello, thanks for reporting the issue.
Looks like act is not providing all context data this action needs.
Most probably ref or repository.default_branch from event payload are missing.
There are no checks for it as those fields should be mandatory.
I will add some default behavior for this case and rest should work.
Otherwise you could probably fix it by providing this values to act using --eventpath.
Should be fixed in few days.

Thanks for your response @dorny.
I have tried your suggestion to emulate the github actions with the following payload:
// .github/workflows/event_push.json
{
    "ref": "refs/heads/test_github_actions",
    "repository": {
        "name": "myproject",
        "full_name": "my-org/myproject",
        "private": true,
        "owner": {
        },
        "default_branch": "develop",
        "master_branch": "master"
    }
}

Then run it: act -j deployToFirebase -v -e .github/workflows/event_push.json
However, I now see a new error:
::error::Unable to locate executable file: git. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

This action uses local installation of git. Container used by act doesn't have it.
Quoting from their README:

These default images do not contain all the tools that GitHub Actions offers by default in their runners.

Looks like git is one of the tools omitted by act
Try to use alternative image with act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04
Originally you also asked:

I am also open to suggestions on how else to test locally to save on the hassle of first sending the code to github and then waiting for the workflow result

Well, as always it depends.
If you have your own build and test scripts, or it's possible to easily invoke them on CLI, You could simply use some local bash script to do the job.
If you need some github actions to execute the build/test then you could create it's own workflow where you would skip all unnecessary actions which works with github API or requires github environment. For example actions which create github status checks or automatically adds labels to PR.
Automatically detecting what files changed and then running the build/test/deploy makes sense if you have many components or you want to avoid maintaining two workflows. This action needs only the refs in event payload and local git so hopefully this was the last issue with this setup.

This action uses local installation of git. Container used by act doesn't have it.
Quoting from their README:

These default images do not contain all the tools that GitHub Actions offers by default in their runners.

Looks like git is one of the tools omitted by act
Try to use alternative image with act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04
Originally you also asked:

I am also open to suggestions on how else to test locally to save on the hassle of first sending the code to github and then waiting for the workflow result

Well, as always it depends.
If you have your own build and test scripts, or it's possible to easily invoke them on CLI, You could simply use some local bash script to do the job.
If you need some github actions to execute the build/test then you could create it's own workflow where you would skip all unnecessary actions which works with github API or requires github environment. For example actions which create github status checks or automatically adds labels to PR.
Automatically detecting what files changed and then running the build/test/deploy makes sense if you have many components or you want to avoid maintaining two workflows. This action needs only the refs in event payload and local git so hopefully this was the last issue with this setup.

Hi @dorny, thanks for the fixes in #40. I have tried using act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04 but the image looks too large (18GB!) and thus running it like this just hangs forever:
act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04 -j deployToFirebase -e .github/workflows/event_push.json
Any ideas, or how I can use a lighter image?

Image needs to be pulled just once. You can do it independently from act: docker pull nektos/act-environments-ubuntu:18.04.
Image size shouldn't affect container startup time, so once you have it locally only remaining issue is it will take 18GB from your disk drive.
Alternative would be to build your own custom image based on the act default image, but with git installed.
If you decide for this solution, you should be sure it's actually worth the effort :)
Building custom image for act is quite out of scope of this project (and my knowledge) so if you will need some support the act repository is probably a better place.

Image needs to be pulled just once. You can do it independently from act: docker pull nektos/act-environments-ubuntu:18.04.
Image size shouldn't affect container startup time, so once you have it locally only remaining issue is it will take 18GB from your disk drive.
Alternative would be to build your own custom image based on the act default image, but with git installed.
If you decide for this solution, you should be sure it's actually worth the effort :)
Building custom image for act is quite out of scope of this project (and my knowledge) so if you will need some support the act repository is probably a better place.

Wonderful, thanks so much Michal that helped. It took a while to install the image but I'm glad it's a one-time thing only

Trying to do the same thing, it looks like the "medium" container now contains git but runs into a problem with github not being a part of the known hosts, PR Ref Issue ref that they're working on.


Mutate logstash convert

Description
This contains logic for handling the conversion from logstash's mutate filter to Data Prepper's various mutate processors
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1042 (1770581) into main (62be34c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main    #1042   +/-   ##
=========================================
  Coverage     91.32%   91.32%
  Complexity      683      683
=========================================
  Files            84       84
  Lines          1994     1994
  Branches        168      168
=========================================
  Hits           1821     1821
  Misses          132      132
  Partials         41       41

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62be34c...1770581. Read the comment docs.


Update rapids JNI and private dependency to 24.06.0

Wait for the pre-merge CI job to SUCCEED

Make it draft until v24.06.0 rapids jni and private jars get released

build


Revert parity package commits for 16 packages

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[x] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
[x] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[x] All source files have up-to-date hashes in the *.signatures.json files
[x] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[ ] Ready to merge

Summary
This reverts 16 parity package commits due to build issues:
AMD64: Build stuck/hang for package gstreamer1-plugins-base
ARM64: Blocked SRPM's containing 9 out of 16 packages
Change Log

Revert 16 parity packages commit
Update licenses entries
update cgmanifest entries

Does this affect the toolchain?
NO
Associated issues
Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-YYYY-XXXX

Test Methodology

Pipeline build id:

ARM64
Buddy Build passes for the packages in question after some build fixes for some packages. https://dev.azure.com/mariner-org/mariner/_build/results?buildId=287119&view=results
[Failure is shown due to 1 missing Runtime requires for a package: "python-qt5-base"]
AMD64
The build for package gstreamer1-plugins-base is stuck and the pipeline ultimately times out. Unable to figure out the reason for the hang issue.
[Local build is successfull]

Please merge after the above pipelines are green.


Sending large packets is slow (on the client)

Bug Report
I've only made a minor modification to the helloworld sample so the message takes bytes instead of a string. Then I send it around 2G of data in a single packet. Tonic seems to spend a significant time (much more then would be the norm for a normal 2GB memcpy) moving this data around.
Ideally it should just zero-copy / move the data into the OS.

Minimal repro attached
tonic-repro.zip
Relates to  #352 as well

Both this and #352 share the same repro case btw.

also @seanmonstar :)


Changes to Task2

Task2- addition of two numbers

@muthyasravani
Task -1
Code verified.
Task -4
Code verified.


Support for cloud certificates

Can a feature be implemented in OpenSSL that enables the use of cloud-based certificates?
https://cloudsignatureconsortium.org/wp-content/uploads/2020/01/CSC_API_V1_<IP_ADDRESS>.pdf

I'm not sure what such a feature would look like.  from reading the document, this doesn't seem like something that belongs in openssl directly, but rather its own application that optionally makes use of openssl for secure communications, something like (SignServer)[https://github.com/Keyfactor/signserver-ce]

In simple words, I know that certifcates are stored today on USB crypto tokens, PKCS7 being used to access the certificate data. The feature scope is to use the cloud certificates in the same manner. For example, encryption could by made using 13.3 Create a remote signature with a credential protected by a PIN and 1.4 credentials/list together with 1.5 credentials/info  retrieve certificates.


Improve db architecture

Surely this cannot be the best way to handle a database...

Also, save more data for an eventual dashboard + more insights. This can also help to fix:

#205
#263


Truenth new lived experience page

address this story:
https://www.pivotaltracker.com/story/show/143547263
can now be merged into develop upon review.

quality of images need improving on bigger browser screen.  will re-open when I have fixed the issue.


Watch the config file for changes

Is your feature request related to a problem? Please describe.
While editing my graphql codegen config I have to restart the watch server every time I make a change.
Describe the solution you'd like
It would be great if the file watcher also responded to changes to the graphql codegen config file!
Additional context
My config is in a file called graphql-codegen.yml, so not in the standard location.

Duplicate of #1735.


chore: Add stories for resource operation select

:tada: This PR is included in version 1.27.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Support source stepping using GitLink

Today I was having a few issues with the library (when actually integrating this into GitTools.Core, which is used by GitLink). However, the library is a black box so it's hard to understand where it's going wrong. Would be great if the source would be "steppable".
For more information, see https://github.com/GitTools/GitLink

btw. let me know if you need help with this.

Hi Geert, pleased to here the SDK is actually beeing used. Not sure what I can do to help you, I haven't signed the the library so please feel free to give me some ideas on how I would make my code "steppable".
I'm not having this issue on my local project where visual studio steps into the SDK seamlessly but maybe visual studio is smart enough to recognize the library is a DLL compiled from another visual studio solution on my hard drive.

If you build the software locally, it will work, but if you reference a NuGet package, it becomes a black box (because there is no source available). The only thing you have to do is between the build and the creation of the NuGet package you should run GitLink:
GitLink.exe c:\source\Jira.SDK -u https://github.com/jonas0007/Jira.SDK -b [branchname]

If you have a local .git directory when doing this, this is sufficient:
GitLink.exe c:\source\Jira.SDK

I have published a new nuget package including the steps you've added above. Can you please test if this works for you?

Don't you have a build script / build server that automatically deploys for you? If not, you might look into AppVeyor (it's free for open source projects if I am not mistaken).

I'm currently running into bug https://github.com/GitTools/GitLink/issues/42 of the GitLink command line so the new version of the nuget package won't be having gitlink support.
I'll look into a build server for my project. AppVeyor seems to be a paying service even for open source projects.

Quote on their front page: "Free for open-source projects."
Make sure you do a full clean before doing a rebuild / running gitlink. What OS / VS language are you using?


use turbo-ws

https://github.com/hugmanrique/turbo-ws

Seems like that one has been archived. Does that mean it's been deprecated?

Mhh yeah you're right, to bad, I was interested by maf's turbo-net library (https://github.com/mafintosh/turbo-net). Anyway, cws that we use here is really performant, let's close this!


feat(#165): Unknown Message

Add Unknown Message class.
Closes: #165

PR-Codex overview
This PR refactors duplicated code in AssertionOfHamcrest and AssertionOfJUnit by creating a new UnknownMessage class. It also adds a license header to UnknownMessage and improves its documentation.
Detailed summary

Refactors duplicated code in AssertionOfHamcrest and AssertionOfJUnit.
Creates a new UnknownMessage class.
Adds a license header to UnknownMessage.
Improves the documentation of UnknownMessage.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@rultor merge

@rultor merge

@volodya-lombrozo OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@volodya-lombrozo Done! FYI, the full log is here (took me 3min)


Example.py: AttributeError: 'Sequential' object has no attribute 'constraints'

Hi guys,
I am getting the following error when trying to run example.py:
Traceback (most recent call last):   File "example.py", line 82, in <module>     agent.compile(Adam(lr=.001, clipnorm=1.), metrics=['mae'])   File "/opensim-rl/lib/python2.7/site-packages/rl/agents/ddpg.py", line 163, in compile     updates = actor_optimizer.get_updates(self.actor.trainable_weights, self.actor.constraints, None) AttributeError: 'Sequential' object has no attribute 'constraints'
Anyone know what this is about? I am using Keras==2.0.8, keras-rl==0.3.0, tensorflow==1.3.0, rl==2.4, osim-rl==1.4.1. Could it be that this was a change in Keras?
EDIT
I now noticed that the function DDPGAgent defined in the rl library takes an actor argument, for which we pass an instance of Sequential from keras.models. Then in DDPGAgent we call that Sequential's constraints attribute. But this doesn't exist. Does anyone know what this rl library is, anyways? I figure it was installed during installation of osim-rl

Ok completely reinstalled everything and now it works. This time it seems like i don't even have an 'rl' library. Slightly confused

rl library is just keras-rl' https://github.com/matthiasplappert/keras-rl/tree/master/rl it seems it might have been some compatibility issue regarding keras-rl`. Closing for now, thanks for reporting!

 Go back to Keras 1 would solve the issue.


Harden error logging

Depending on the exception, the error logging itself could fail.

This resolves the direct issue of #252.  Separately, we should also process tiff files that end in .svs as tiff files if they do not have an Aperio ImageDescription.