Add diagnostic tests

Pull Request Checklist:

[x] This PR addresses an already opened issue (for bug fixes / features)

This PR fixes #424

[x] Tests for the changes have been added (for bug fixes / features)
[x] Documentation has been added / updated (for bug fixes / features)
[x] HISTORY.rst has been updated (with summary of main changes)

[x] Link to issue (:issue:number) and pull request (:pull:number) has been added.

[ ] bumpversion patch has been called on this branch
[x] The relevant author information has been added to .zenodo.json

What kind of change does this PR introduce?

The goal of this PR is to introduce diagnostic tests for sdba, through properties.py and measures.py.
A diagnostic test calculates a statistical property on a reference dataArray and a prediction dataArray, then calculates a measure of the difference.
Properties collapse the time dimension of the input and output one value per grid point.
An example is shown in sdba-advanced.ipynb.
Properties include : 'mean', 'var', 'skewness', 'quantile', 'spell_length_distribution', 'acf', 'annual_cycle_amplitude', 'annual_cycle_phase', 'corr_btw_var', 'relative_frequency', 'trend', 'return_value'
Measures include : bias, relative bias, ratio, circular bias, rmse, mae, annual_cycle_correlation

Does this PR introduce a breaking change?
No
Other information:

Coverage increased (+0.2%) to 91.008% when pulling a21f412d0fac32ece0a1eae863b16d5a1343f7b7 on add-diagnostic-tests into bd10c799d885531a0061a15f04a4b90bd9c32a48 on master.

Looking at the doc I saw that some "See also" sections were not rendering correctly. And I realized the syntax I suggested was not the correct one, sorry! See here : https://numpydoc.readthedocs.io/en/latest/format.html#see-also
No :py:func: needed, and a description is possible.

Looking at the doc I saw that some "See also" sections were not rendering correctly. And I realized the syntax I suggested was not the correct one, sorry! See here : https://numpydoc.readthedocs.io/en/latest/format.html#see-also No :py:func: needed, and a description is possible.

I think the issue was that I put the wrong link in the intersphinx_mapping... I think it shows up correctly in the latest commit ( with :py:func:.

I'm currently testing this and I think we must adapt the properties to the indicator model. One reason is the current handling of output attributes : you often use the long_name of the source, but it sometimes doesn't always exist and the call fails. Instead of finding a workaround, it would be just simpler to use the indicator pipeline for the attributes and checks.
I think we could achieve this by moving all properties functions to _properties.py, creating a new subclass of Indicator in properties.py and then creating the indicators in the same file. There will be issues with the modules, registry and other stuff... I can have a look at this if you want.

I see the issue. What does the indicator class do when long_name doesn't exist ?
I can try to do it and let you if/when I have issues.

Well, the indicators don't do anything about the long names of the inputs, or about any other attributes in fact... Hum.
I suggest we don't do anything in this PR and, for the moment, drop this formatting in the long names. I'll open an issue about extending Indicator.format to handle the attributes of the inputs.

So we don't put any long_name for the outputs or one very vague that doesn't represent the input?

I am indeed suggesting we put only a vague long name for now, so this PR can be merged and used in ESPO-R/G.

While testing this I stumbled upon the same performance problems as for the other xclim indices. However, here a lot of the indices are using groupby instead of only resample this makes them candidate to the use of map_groups which I used in the adjustments to simplify the dask load.
I am currently trying to apply this logic to the slowest properties : spell_length_distribution, return_value and maybe others.

Here are my accelerations:

spell_length_distribution,  acf and return_value were adapted using map_groups. This way, dask sees a single task for each non-temporal chunk.
numbization of rle_1d : this greatly accelerated spell_length_distribution.

I had to change Grouper to accept "time.season" and workarounds a frustrating bug where the coordinate is returned in alphabetical order instead of the correct one...
acf is still at least 5 times slower than spell_length. return_value is frustratingly slow and I abandoned the idea of using it for ESPO-R, at least for now. But, when running before the map_groups addition, the creation of the variables (not the computation!) would take more than 10s each time with my data. Now it runs in less than 150 ms.
As we use the Grouper and the input argument group in the rest of sdba, what would you say we use it here? Instead of time_res='season', we would pass group='time.season'.

When I run the 3 modified function with time_res='season', I get a Not Implemented error.
Not sure yet how the map_groups decorator really work yet, will look into it more tomorrow.
Traceback (most recent call last):
  File "/exec/jlavoie/.conda/xclimDev/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3457, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-14-f037ee1ef75d>", line 1, in <module>
    out = xc.sdba.properties.return_value(sim.pr, time_res='season')
  File "<boltons.funcutils.FunctionBuilder-304>", line 2, in return_value
  File "/home/jlavoie/development/xclim/core/formatting.py", line 336, in _call_and_add_history
    outs = func(*args, **kwargs)
  File "/home/jlavoie/development/xclim/sdba/properties.py", line 753, in return_value
    out = frequency_analysis_method(
  File "<boltons.funcutils.FunctionBuilder-308>", line 2, in _map_blocks
  File "/home/jlavoie/development/xclim/sdba/base.py", line 445, in _parse_group
    return func(*args, **kwargs)
  File "/home/jlavoie/development/xclim/sdba/base.py", line 585, in _map_blocks
    coords[group.prop] = group.get_coordinate(ds=ds)
  File "/home/jlavoie/development/xclim/sdba/base.py", line 185, in get_coordinate
    raise NotImplementedError()
NotImplementedError

When I run the 3 modified function with time_res='season', I get a Not Implemented error.

Oh I didn't see your comment. Hmm. It's like you have the changes in properties,.py but not those in base.py? Weird! It works on my side and I see the changes here... Can you try with the latest changes?

Ok the issue was with my remote deployment on pycharm. It was automatically syncing properties.py, but not base.py. It works now!

@Zeitsperre I applied your suggestions, but doctest doesn't like the "path_to_tas_file"/ "path_to_pr_file"... It gives a file not found error. any idea why?

One of the doctests has been running for several minutes now... Are there any examples that are particularly heavy on computation?

return_value is probably the heaviest. Should I remove the example ?

return_value is probably the heaviest. Should I remove the example ?

I'm going to try profiling the tests and push some changes if needed in another branch. I don't think it was one of yours. Feel free to merge!


Not able to access static methods of parent class from child class when inheriting with generic types.

Not able to access static methods of the parent class from the child class when inheriting with generic types.
@model('ChildModel')
export class ChildModel extends ExtendedModel(() => ({
    baseModel: modelClass<ParentModel<ChildModel>>(ParentModel),
    props: {
        name: prop<string>()
    }
})) {}

@model('ParentModel')
export class ParentModel<T> extends Model(<T>() => ({
    id: prop<string>()
}))<T> {

    @modelAction
    static doSomething(id: string) {
        // TODO
    }

}

While trying to pass generic types I am not able to call doSomething on ChildModel. Typescript gives the error Property 'doSomething' does not exist on type 'typeof ChildModel'.ts(2339)
However, if I don't use Generics:
@model('ChildModel')
export class ChildModel extends ExtendedModel(ParentModel, {
        name: prop<string>(),
    }
) {
}

It works fine and can access doSomething on ChildModel. (ChildModel.doSomething())

I've been looking into it, but unfortunately I can't fix it. As soon as I add any extra static property the trick that makes generic works breaks badly and there seems to be no way in TS to do so (or at least I can't find any). :(
I recommend to either call the static using ParentModel.doSomething() or just move doSomething out of the class itself.


bug(Overlay): When attaching an overlay to the DOM, the whole page flickers

Reproduction
Steps to reproduce:

create an ng-template with cdkPortal directive
create an OverlayRef with Overlay
attach the template ref (CdkPortal) to the OverlayRef

Expected Behavior
When I call this sequence of actions I expect to see a modal without causing the page to flicker.
Actual Behavior
for a fraction of a second all texts are invisible on the page
which creates and effect of flickering, the texts are there in the DOM, but not visible
then the PortalOverlay is shown. This happens only the first time I open a CDKPortal
Subsequent call do not provoke this issue.
Looking at the page source initially the cdk-overlay-container is not present in the DOM, it stays there after I close the "modal". Happens the same with MatDialog

Digging deeper
createEmbeddedView(templateRef, context, index) {
                /** @type {?} */
               -> const viewRef = templateRef.createEmbeddedView(context || (/** @type {?} */ ({})));
                this.insert(viewRef, index);
                return viewRef;
            }

this line cause the whole texts to disappear.
At this moment all texts are invisible.
Further researching and creating base templateRef with a an ng-container as ViewContainerRef, the insert method cause the whole app to flicker.
So this must be a bug in Angular
Environment

Angular: 9.1.9
CDK/Material: 9.2.4
Browser(s): Chrome latest
Operating System (e.g. Windows, macOS, Ubuntu): Windows 10

Can you post a Stackblitz that shows the issue? We have several components that use the same pattern that you're describing and they don't cause any flickering.

I have a stackblitz, where I don't see the issue. I'm trying to pinpoint reason I don't see that consistently. Let me research a bit more. I know you are using the same pattern in multiple places, I wonder why ng-boostrap is not using that pattern for their modals.

Ok, I think I have found the issue. It might be related with that my modal scss is loading the whole list of font-faces again, thus triggering the loading of the fonts again, which explains why it doesn't happen on second modal open, maybe because fonts requests were cached, and maybe explains the invisibility of the texts, but not on any other element or component and the text of a simple button without a font set was not disappearing.


Multiple command produce... when using the test plugin

Hello I just added the Test Plugin to my UnitTests as a Build Tool Plug-in.
However I get this error saying that multiple command produce the Unit Tests product.
Am I doing something wrong in the setup?
I have added in the project Prefire, incuded it in the base target, and then as a build tool plugin only for the tests (I do not need the Playbook)

Solved seems that the problem was that the tests were not able to get generated in derived data, by using the yml and specifying an internal project location the problem was fixed.


Brief white background whilst scrolling from last image to first

In a 2 image scroller, using background images, when the slider moves from the last image to the first image, it scrolls to a white background until the first image snaps into view. I have also tried using more than 2 images with the same issue between the very last and first slide.
https://jsfiddle.net/f7p5toan/

Tested on Chrome, Chrome for Android and Edge
jQuery 3.4.1 and jQuery 1.11.0 tested with Slick 1.8.1

It's because you're using css id's. A css id can only exist once per page so slick can't clone it. Use css classes instead and it will work.
https://jsfiddle.net/6pjk7209/5/

Perfect that worked thank you


test(publish): Cover more wait-for-publish cases

These came from trying to guess what cases are causing problems in #11314.  Unfortunately, can't reproduce it so far but figured it'd be good to keep these around.

Thanks!
@bors r+

:pushpin: Commit d87966b4661889d82770ba92dcd4c60de9363ae7 has been approved by ehuss
It is now in the queue for this repository.

:hourglass: Testing commit d87966b4661889d82770ba92dcd4c60de9363ae7 with merge 68e9d6c17b9c12baf4c42fa1d15a91556cfb61c4...

:sunny: Test successful - checks-actions
Approved by: ehuss
Pushing 68e9d6c17b9c12baf4c42fa1d15a91556cfb61c4 to master...


Support deleting files form browser download folder

Now selenoid supports getting file from browser download folder by internal http server from url GET http://${selenoidHost}:4444/download/${sessionId}/${filename}
It will be nice to allow remove file by
DELETE http://${selenoidHost}:4444/download/${sessionId}/${filename}

From my point of view, it will be nice to document this feature here:
https://aerokube.com/selenoid/latest/

Done. https://aerokube.com/selenoid/latest/#_accessing_files_downloaded_with_browser


Add list element

A list can bei either ordered or unordered and contains a set of paragraphs.

Coverage remained the same at 100.0% when pulling f03bcd96c1e110156371acf993b4251fa400519b on dsager:list-element into 0ae2a5d3df82aef6f8d969ca4e8ffba4dc0becbf on Devex:master.

+1


JavaScript IR Support

Resolves #60

Functions that uses Kotlin DataTime library need to be annotated with @ExperimentalTime as not doing so was resulting in a build error.

This declaration is experimental and its usage must be marked with<EMAIL_ADDRESS>or '@OptIn(kotlin.time.ExperimentalTime::class)'

@twyatt do we need PR #65 to be merged before merging this PR?

@twyatt do we need PR #65 to be merged before merging this PR?
Getting the following build error

Execution failed for task ':coroutines:signAndroidDebugPublication'.
The key ID must be in a valid form (eg 00B5050F or 0x00B5050F), given value:

I think it will fix your build issues. I'll get it ready and pull it out of draft.

@KVelasco just a heads up, did an Update branch on this PR as I wanted to get it in a compiling state to test with some other things on my local machine. Appologies.

Please disable binary-compatibility-validator for now:
    // Breaks with:
    // A problem occurred configuring project ':registration'.
    // > org.gradle.api.internal.tasks.DefaultTaskContainer$DuplicateTaskException: Cannot add task 'apiBuild' as a task with that name already exists.
    //
    // Disabling until https://github.com/Kotlin/binary-compatibility-validator/issues/38 is fixed.
//    id("binary-compatibility-validator") version "0.6.0"


[heatmap] add  &  fixes #47

Also fixing #47
[shape] refactor callOrValue => additionalProps
[mock data] add genBin() + genBins() for heatmapping

Haven't published @vx/heatmap yet so build fails. Will merge and publish v0.0.115 from master.


provide visibility into final prompt

For debugging or other traceability purposes it is sometimes useful to see the final prompt text as sent to the completion model.
It would be good to have a mechanism that logged or otherwise surfaced (e.g. for storing to a database) the final prompt text.

this should be possible with tracing! have you tried it out? https://langchain.readthedocs.io/en/latest/tracing.html

I kept an eye on tracing debug system documentation, nevertheless a minimal requirement could be to just have some "very verbose" flag for llms and/or chains, to print out the LLM prompts (+ completions).
Consider the following chunk:
llm = OpenAI(temperature=0)

template='''\
Please respond to the questions accurately and succinctly. \
If you are unable to obtain the necessary data after seeking help, \
indicate that you do not know.
'''

prompt = PromptTemplate(input_variables=[], template=template)

llm_weather_chain = LLMChain(llm=llm, prompt=prompt, verbose=True)

tools = [Weather, Datetime]

agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)

The output of the above program shows the agent behavior, in great colorized texts.

but it doesn't show the text prompts sent to the LLM (and the completions feedback). This is what I miss more. I'd like to have a llm() argument verbose=True to have llm interaction printed too.

BTW, what the chain flag verbose=True does? I do not see nothing printed.

Thanks
giorgio

there is a verbose flag you can pass into the llm! llm = OpenAI(temperature=0, verbose=True) should print it out

Thanks Harrison,
I set the LLM verbose flag to true, but I don't see the LLM prompt+completion printed out.
Instead the Agent verbose flag works as expected:
$ cat agent.py
#
# tools_agent.py
#
# zero-shot react agent that reply questions using available tools
# - Weater
# - Datetime
#
# get the question as a command line argument (a quoted sentence).
# $ py tools_agent.py What about the weather today in Genova, Italy
#
import sys

from langchain.agents import initialize_agent
from langchain.llms import OpenAI
from langchain import LLMChain
from langchain.prompts import PromptTemplate

# import custom tools
from weather_tool import Weather
from datetime_tool import Datetime

llm = OpenAI(temperature=0)

template='''\
Please respond to the questions accurately and succinctly. \
If you are unable to obtain the necessary data after seeking help, \
indicate that you do not know.
'''

prompt = PromptTemplate(input_variables=[], template=template)

# debug
# print(prompt.format())
# sys.exit()

# Load the tool configs that are needed.
llm_weather_chain = LLMChain(
    llm=llm,
    prompt=prompt,
    verbose=True
)

tools = [
    Weather,
    Datetime
]

# Construct the react agent type.
agent = initialize_agent(
    tools,
    llm,
    agent="zero-shot-react-description",
    verbose=True
)

if __name__ == '__main__':
    if len(sys.argv) > 1:
        question = ' '.join(sys.argv[1:])
        print('question: ' + question)

        # run the agent
        agent.run(question)
    else:
        print('Agent answers questions using Weater and Datetime custom tools')
        print('usage: py tools_agent.py <question sentence>')
        print('example: py tools_agent.py what time is it?')

Run-time:
$ py agent.py "how is the weather today in Genova?"
question: how is the weather today in Genova?

> Entering new AgentExecutor chain...
 I need to get the weather forecast for Genova
Action: weather
Action Input: {"when":"today","where":"Genova"}
Observation: {"forecast": "sunny", "temperature": "20 degrees Celsius"}
Thought: I now know the weather forecast for Genova
Final Answer: The weather today in Genova is sunny with a temperature of 20 degrees Celsius.

> Finished chain.

Thanks Harrison,

I set the LLM verbose flag to true, but I don't see the LLM prompt+completion printed out.
Also the LLMChain verbose flag seems doing nothing ?!
Instead the Agent verbose flag works as expected, printing nice coloured traces.

$ cat agent.py
#
# tools_agent.py
#
# zero-shot react agent that reply questions using available tools
# - Weater
# - Datetime
#
# get the question as a command line argument (a quoted sentence).
# $ py tools_agent.py What about the weather today in Genova, Italy
#
import sys

from langchain.agents import initialize_agent
from langchain.llms import OpenAI
from langchain import LLMChain
from langchain.prompts import PromptTemplate

# import custom tools
from weather_tool import Weather
from datetime_tool import Datetime

llm = OpenAI(temperature=0, verbose=True)

template='''\
Please respond to the questions accurately and succinctly. \
If you are unable to obtain the necessary data after seeking help, \
indicate that you do not know.
'''

prompt = PromptTemplate(input_variables=[], template=template)

# debug
# print(prompt.format())
# sys.exit()

# Load the tool configs that are needed.
llm_weather_chain = LLMChain(
    llm=llm,
    prompt=prompt,
    verbose=True
)

tools = [
    Weather,
    Datetime
]

# Construct the react agent type.
agent = initialize_agent(
    tools,
    llm,
    agent="zero-shot-react-description",
    verbose=True
)

if __name__ == '__main__':
    if len(sys.argv) > 1:
        question = ' '.join(sys.argv[1:])
        print('question: ' + question)

        # run the agent
        agent.run(question)
    else:
        print('Agent answers questions using Weater and Datetime custom tools')
        print('usage: py tools_agent.py <question sentence>')
        print('example: py tools_agent.py what time is it?')

$ py agent.py "how is the weather today in Genova?"
question: how is the weather today in Genova?

> Entering new AgentExecutor chain...
 I need to get the weather forecast for Genova
Action: weather
Action Input: {"when":"today","where":"Genova"}
Observation: {"forecast": "sunny", "temperature": "20 degrees Celsius"}
Thought: I now know the weather forecast for Genova
Final Answer: The weather today in Genova is sunny with a temperature of 20 degrees Celsius.

> Finished chain.

good catch - we need to fix this bug probably, but currently the wya to do it would actually be to set
agent.agent.llm_chain.verbose=True

Thanks. The workaround works, but yes I think it's a bug.

I am studying the project and wanted to do some contributions, fixing some bugs/issues might be a good start,  so I read through this issue and related code, I think the issue happens because there are actually two chains:

an outer one(AgentExecutor is a chain) returned by  initialize_agent(...)
an inner one created when initializing the agent, this inner one is the chain that really talks to the LLM

initialize_agent(..., verbose=True) just makes the outer chain verbose, and the workaround agent.agent.llm_chain.verbose=True is for the inner chain
With the above analysis, I think there might be 2 ways to fix the issue:

when initialize_agent is called with verbose=True, we also set the inner chain verbose, this makes the API simple and easy to understand, the problem is that the API's output might be over verbose
call initialize_agent like initialize_agent(..., agent_kwargs={"verbose": True}, verbose=True), the agent_kwargs is for the inner chain, we can use it to control the inner chain's output, this way, we can have precise control for the output, but the API might be a little bit confusing

@hwchase17 do you have some suggestions?

Thanks!
I'd add a note on the functional meaning of "verbose"

when referred to a LLM instance, the expected behavior (in my mind) is to show ALL interactions (prompt+completion) with the LLM.
when referred to an agent instance, the expected behavior is the working logs that explain what the agent is doing.

so, i see the LLM verbosity as something different (at a lower level) from the agent verbosity.
Does it make sense?

maybe adding a verbose_llm parameter to initialize_agent will make the API easy to understand

Well, it could be a way, but currently, when you set verbose==True I see these different cases:

llm => does nothing (a bug ?)
chain => does nothing because the bug (?): prompt+completion logs print
agent => the agent behavior logs print

So llm, chain, agent already have their own distinct verbose flag with possible different meaning. I'd prefer to leave the same flag name "verbose". What is not fully clear is what means verbose for each of llm, chain, agent.

Yep, now impossible to see final executed promst :(

Is there any update on this? I think it is critical to be able to see the final prompt sent to the LLMs. Currently working with Langchain is too obscure, it makes it really difficult to build complex chains without making mistakes.

having the same issue, I need to see the final prompt too.

It looks like one possible workaround the get the final prompt is to attach a StdOutCallbackHandler to the chain
handler = StdOutCallbackHandler()
chain.run(... , callbacks=[handler])

Setting agent.agent.llm_chain.verbose=True worked for me with the latest version (langchain==0.0.216). I agree that expected behavior is that setting verbose=True on the LLM would do this.

Modifying debug=True will print every prompt agent is executing though may not be as pretty as verbose.
import langchain
langchain.debug=True
response = agent.run(prompt)
langchain.debug=False

I think verbose is designed to be on higher level for individual queries but for debugging and granular control debug is more useful.

Using langchain (0.0.256)
Below logs the message to the console:
import langchain
langchain.verbose=True

If you want the prompt as a variable, I'd suggest using callbacks:
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.callbacks.base import BaseCallbackHandler

class MyCustomHandler(BaseCallbackHandler):
    def on_chain_start(self, serialized, inputs, **kwargs):
        print(serialized)

handler = MyCustomHandler()
llm = OpenAI()
prompt = PromptTemplate.from_template("1 + {number} = ")

chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler])
chain.run(number=2)

Hopefully this helps!

Is there an update to this? On top of final prompt, i believe the final response coming from OpenAI would be helpful, things like prompt token count, completion token, stop reason, etc.

If anyone is looking for a simple string output of a single prompt, you can use the .format() method of ChatPromptTemplate, but should work with any BaseChatPromptTemplate class.
I struggled to find this as well. In my case I wanted the final formatted prompt string being used inside of the API call.
Example usage:
# Define a partial variable for the chatbot to use
my_partial_variable = """APPLE SAUCE"""

# Initialize your chat template with partial variables
prompt_messages = [
    # System message
    SystemMessage(content=("""You are a hungry, hungry bot""")),
    # Instructions for the chatbot to set context and actions
    HumanMessagePromptTemplate(
        prompt=PromptTemplate(
            template="""Your life goal is to search for some {conversation_topic}. If you encounter food in the conversation below, please eat it:\n###\n{conversation}\n###\nHere is the food: {my_partial_variable}""",
            input_variables=["conversation_topic", "conversation"],
            partial_variables={"my_partial_variable": my_partial_variable},
        )
    ),
    # Placeholder for additional agent notes
    MessagesPlaceholder("agent_scratchpad"),
]

prompt = ChatPromptTemplate(messages=prompt_messages)
prompt_as_string = prompt.format(
    conversation_topic="Delicious food",
    conversation="Nothing about food to see here",
    agent_scratchpad=[],
)

print(prompt_as_string)

System: You are a hungry, hungry bot
Human: Your life goal is to search for some Delicious food. If you encounter food in the conversation below, please eat it:
###
Nothing about food to see here
###
Here is the food: APPLE SAUCE

I ended up using callbacks (like StdOut / self-implemented loguru-based / langfuse / arize-phoenix / mlflow / wandb)

Wait, what is the final solution for this though? I can't wrap my head around making things complex for something that should have been basic.

@krishna-praveen for me it is usage of community-provided / self-implemented langchain callback mechanism

chain.prompt.format_prompt(**input)

For an uncompromising solution, hijack/monkey-patch the llm.client.create method to save what LangChain is going to send to the OpenAI/LLM API. For funsies (probably not for production), an implementation and demo is in my notebook here.

Yeah, trying to find a solution for this as well, on the JS version.
I have an agent with tools, and I'd like to see a final prompt that ChatGPT will receive.
The chain.prompt.format({...inputs}); will only print the version of the prompt that my code provides, that's clearly not the entirety of it.

The chaining concept is a little difficult to get visibility into - I solved with the following:
def debug_lambda(x):
    print(x)
    return x

debug_runnable = RunnableLambda(debug_lambda)

This means you can simply pipe through your debug runnable to see the prompt that goes through:
ie.
                      | self.prompt
                      | debug_runnable
                      | self.model

Investigating a final prompt is such a common thing that it seems like there should be a nicer way of doing this.


The video's look weird

all the stuff keeps pushing up into the Video's region

I can replicate this, seems 2.1.7 is finally total broken.

Seems Youtube is updated recently, everything goes wrong for me too just a few minutes ago.

Yeah, youtube updated and changed a lot about the way the UI is figured out. It's a mess.

Yeah I'm seeing a shifted player window when using the 720p player size.

Related
#1695
#1696

Yeah, i'm experiencing the same issue in the dev version as well. @YePpHa could you please fix this? :smiley:

I am also having this issue.

Yep, apparently caused by a style update. But the update has finally broken the stable build and it should really be taken down and users should be asked to use the developer build at all times :/


vidstige/jadb#34 implemented

#34 implemented

Great! Can you explain a bit more on how this works and the benefit? I'm no Maven expert here..

For future, please add a descriptive commit message, rather than just linking to github issuse. You may link it as well of course 👍

Comment about commit description is just - will write more about code
What about maven:

Dependencies

Maven stores dependencies in special repositories (for example https://mvnrepository.com/)
Each dependency has groupId, artifactId and vesion - and dependencies are defined in pom.xml - xml file which in declarative style describe structure of the project

Assemble process in maven

Assemble process in maven has lifecycle stages. Plugins (which are also defined in pom.xml and also have versions) can add some additional actions and steps

Version plugin is a plugin which can:

Create report about all  new versions available in maven repositories for both dependencies and maven plugins
Update them himself (mvn versions:use-latest-versions and so on) automatically

So I think this plugin is not vital but make process of updating easier =)
P.S. For example with have not the latest version of junit - by typing mvn versions:use-latest-versions in terminal you are able to get latest easily

Alright, cool! Could you perhaps add a short text, or even a script, with how one would build and update the package in a repository? Perhaps you can just plop the command line you use yourself in a .sh file or such?

sh script may be not the best solution - cause maintainers may use Windows for example where sh files will not work
May write a wiki page, or put comments in pom.xml file of how to use this plugin

Yeah, but at least they can open the sh file and perhaps guess how to run the command on windows. Putting it in either README.md or a new .md file also works. Comment in actual pom.xml also sounds great. There is no wiki for simplicity right now.

Added to pom.xml

Thanks man


Default UIPanel starting_layer_height to 1

Adds a default value of 1 to the starting_layer_height  parameter in the pygame_gui.elements.UIPanel  constructor.
This should be consistent with all other element classes in the pygame_gui.elements package, which default this parameter to 1 (if exposing it as an init argument).
Resolves issue #356.

I think I originally left it without a default value because I kept having to mess with the starting height on UIPanels in my games. Usually because I was using some 'in-world' UI and I was using the panels as containers for HUD elements that were supposed to be above - e.g. floating health bars.
But I suspect it is better to be consistent with everything else.
Thanks for the PR!


fix toggle error from qtwidgets dependency

this dependency add some bugs due to Pyside, now it's replaced with native qtpy

Codecov Report
Attention: 361 lines in your changes are missing coverage. Please review.

Comparison is base (cee6071) 13.10% compared to head (134ae08) 11.49%.
Report is 25 commits behind head on main.

Files
Patch %
Lines

src/napari_omaas/_widget.py
1.96%
249 Missing :warning:

src/napari_omaas/utils.py
20.00%
112 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #46      +/-   ##
==========================================
- Coverage   13.10%   11.49%   -1.62%
==========================================
  Files           8        8
  Lines        1381     1610     +229
==========================================
+ Hits          181      185       +4
- Misses       1200     1425     +225

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Set icon on generic OAuth / OIDC Identity Providers

Preflight Checklist

[X] I could not find a solution in the existing issues, docs, nor discussions
[X] I have joined the ZITADEL chat

Describe your problem
I have a custom identity provider setup and it looks out of place without an icon like the google, github, etc. auth buttons have.
Describe your ideal solution
I'd like to be able to either upload or include a link to an image within the identity provider settings, that will be displayed on the registration and login pages.
Version
No response
Environment
Self-hosted
Additional Context
No response

Hi @thepaperpilot
thanks for adding an issue, but we already have one for the same improvement.
To avoid duplicates I will close this one here.
Would be great if you can leave a reaction on the other issue so we are able to track how many are interested in it.
https://github.com/zitadel/zitadel/issues/5455


Live

Docs Build status updates of commit b005de4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

windows-iotcore/TOC.md
:white_check_mark:Succeeded

windows-iotcore/develop-your-app/MultiLang.md
:white_check_mark:Succeeded

windows-iotcore/developer-tools/python.md
:white_check_mark:Succeeded

windows-iotcore/learn-about-hardware/HardwareCompatList.md
:white_check_mark:Succeeded

windows-iotcore/learn-about-hardware/iotnxp.md
:white_check_mark:Succeeded

windows-iotcore/media/Python/global_sign_cert.png
:white_check_mark:Succeeded

windows-iotcore/windows-iot-core.md
:white_check_mark:Succeeded

windows-iotcore/windows-iot-enterprise.md
:white_check_mark:Succeeded

windows-iotcore/windows-iot.md
:white_check_mark:Succeeded

windows-iotcore/windows-server.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


capability io.quarkus.rest.client provided by multiple providers

Hi,
I'd love to use your project in a project of mine, but migrated to resteasy-reactive, as that seemed to recommended by the quarkus documentation (https://quarkus.io/guides/resteasy Quote: It is now recommended to use RESTEasy Reactive ....). However that creates a conflict:
Please make sure there is only one provider of the following capabilities:
capability io.quarkus.rest.client is provided by:
  - io.quarkus:quarkus-rest-client-reactive::jar:2.9.2.Final
  - io.quarkus:quarkus-rest-client::jar:2.9.2.Final

I tried to exclude in gradle, hoping for a drop-in replacement:
    implementation ('io.quarkiverse.openapi.generator:quarkus-openapi-generator:0.9.0') {
        exclude group: 'io.quarkus', module: 'quarkus-rest-client'
    }

But that did not work as well. Any suggestions on how to proceed?
Thx
boun

@boun, I believe we need to do this within the extension itself since there are changes in the deployment and runtime. You won't be able to change the dependencies this way. Additionally, I believe we might need to change the templates as well. I can work on this a few weeks from now.

@boun I have started the migration to RESTEasy Reactive.
This is the tracking issue: https://github.com/quarkiverse/quarkus-openapi-generator/issues/150

Closing as this is being tracked by https://github.com/quarkiverse/quarkus-openapi-generator/issues/150


Default read response format will be changing soon

When Validator PR#258 lands the JSON structure of the response to read queries will change.
#258 also includes 3 optional params that enable the caller to specify what format they would like to get back.
This SDK will need to expose read method options that can be set be the caller, and we will need to do a major version bump with the new response structure.

related: https://github.com/tablelandnetwork/js-tableland-cli/issues/63

This is fixed by #158


add service_url and acceptssl options for AppScan 360

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


lodash: changed _.isFunction() method

https://lodash.com/docs#isFunction
https://lodash.com/docs#_ (description of the chainable wrapper)

@chrootsu thanks mate!


UX Researcher: Creating Research Questions/Research Plan Template Story

As a UX Researcher I want to create a research plan with clear overarching research questions so that I know what types of questions I need to ask in my interviews to gather the right kinds of data from interviews.
Feature: Create research plan and research questions for interviews
  As a UX researcher
  I want to create a clear research plan with clear research questions I want answered
  So that I can create interview questions that will get me the data I need

Scenario: Create research questions
    Given that our internal IBM users will be the main user of the WOX CMS
      And that we do not understand their workflows, needs and requirements
     Then I should list out clear research questions that will lead the direction in creating our research plan and interview questions

  Scenario: Craft research plan
    Given that I will have crafted clear and concise research questions
      And that I will be using those questions to guide creation of my research plan
     Then I will create a research plan for each CMS user research topic I am tackling

  Scenario: Craft interview script
      Given that I will have created research questions and a research plan
      And that I will be using those assets to guide my interviews
     Then I will create an interview script/interview questions that will help us gather the data we need to help answer the questions we have about our users

[ ] Create research questions
[ ] Create research plan
[ ] Create interview script

@ineedsubstance
Template story we can use to base our research topic research plan creation stories around.

this looks good 👍

Thanks for reviewing!


node: commit package-lock.json

Mentioned in #2060 as well as by @panicstevenson several times.  I just ran npm install locally and committed what was there.


How to use other models

Is it possible to use updated rife models like 4.13 with this?

Of course, you can replace the RIFE-related files in the models folder, such as IFNet_HDv3, with the latest ones, and simply change the import statement.
Finally, change the latest weight path here


Error: Path must be a string.  Recieved undefined. when editing unsaved doc

VSCode Version: Code 1.11.2 (6eaebe3b9c70406d67c97779468c324a7a95db0e, 2017-04-13T07:56:42.517Z)
OS Version: Darwin x64 16.5.0
Extensions:

Extension
Author
Version

vscode-terminal-here
Tyriar
0.0.5

clojure
avli
0.5.3

file-templates
brpaz
1.1.0

vscode-eslint
dbaeumer
1.2.8

json-tools
eriklynd
1.0.2

Lisp
mattn
0.1.7

prettify-json
mohsen1
0.0.3

Ruby
rebornix
0.12.1

vscode-flow
rtorr
276.3.0

rails-go-to-spec
sporto
0.0.1

Steps to Reproduce:

open a new document, do not save it
Change the text highlighting to Javascript or Javascript React
Start typing.  This error gets rethrown repeatedly.

This might be an issue with an outdated eslint installation. Pls see #13598.

@Alphy11 this is a problem in the eslint node module (not the ESLint VS Code extension). You need to update the eslint node module.
Here is what I see with ESlint 1.2.8 and eslint 3.12.2

Please ping if you can still reproduce after upgrading.


fix postgresql missing SQL

Please separate out features and fixes into individual pull requests
and reference the issue if it exists.

Thanks!


update six dependancies for new collections moves

Related to:
https://github.com/pytest-dev/pytest-flask/issues/106


[CodeCompletion] Report solutions from the result builder transform

Previously, we never reported any solutions through the sawSolution callback when applying the result builder transform. This caused us to always end up in the fallbackTypeCheck case for code completion, effectively type checking the constraint system twice.
With this change, we call sawSolution from the result builder transform, reducing the type code completion takes in SR-14651 from ~2.1s to ~1.4s (measured using swift-ide-test with assertions enabled).
Fixes rdar://78297973 [SR-14651]
Resolves rdar://77673447

@swift-ci Please test

@swift-ci Please SourceKit stress test

@swift-ci Please test

@swift-ci Please SourceKit stress test

I just did a couple more measurements regarding code completion performance with this change and while it improves performance in some situations by ~30%, I saw a >1500x performance regression in other cases.
The pattern I’ve been able to find so far is that this PR improves performance in small (~10 lines) result builders but regresses for unresolved dot completions of larger (~30 lines) result builders.
In any case, I’m closing this PR for now as it’s not the easy fix that I hoped it would be. We need to come up with a more complete answer to address performance improvements in result builders in the future.


[BACKLOG-39875] Change date format in logging console output

:white_check_mark: Build finished in 42m 37s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:ok_hand: All tests passed!
Tests run: 5338, Failures: 0, Skipped: 14    Test Results

:information_source:  This is an automatic message

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 42m 41s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:ok_hand: All tests passed!
Tests run: 5338, Failures: 0, Skipped: 14    Test Results

:information_source:  This is an automatic message

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 45m 48s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:ok_hand: All tests passed!
Tests run: 5338, Failures: 0, Skipped: 14    Test Results

:information_source:  This is an automatic message

:white_check_mark: Build finished in 45m 32s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core,plugins/excel/core

:ok_hand: All tests passed!
Tests run: 5356, Failures: 0, Skipped: 14    Test Results

:information_source:  This is an automatic message

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 44m 10s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:ok_hand: All tests passed!
Tests run: 5356, Failures: 0, Skipped: 14    Test Results

:information_source:  This is an automatic message

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 42m 35s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core

:ok_hand: All tests passed!
Tests run: 5356, Failures: 0, Skipped: 14    Test Results

:information_source:  This is an automatic message


Artifact in code block (6.2.2)

6.2.2 Map

This is another one that seems to be ok since we moved to A5.


Fix issue where --enable-remote-node-identity=false causes policy drops

This PR fixes #10702 by reworking the --enable-remote-node-identity=false behaviour to ensure that it injects BPF policymap entries that allow traffic from the remote-node identity if the policy generation would otherwise generate BPF policymap entries that allow traffic from the Host identity.
In other words:

If --enable-remote-node-identity=false, then remote identity traffic will be allowed by default, like in v1.6.
If --enable-remote-node-identity=true, then remote identity traffic will be dropped by default, like in existing v1.7 releases.

Consider reviewing commit-by-commit. It is structured as follows:

Patch 1 refactors some L7 wildcard handling to make it more generic for host and remote node identity wildcarding.
Patch 2 is the core policy generation logic change. It's pretty simple but builds on the existing policy generation logic to handle the allow of the remoteNode identity both for the automatic allow from host case, and also when explicit from-host policy is injected.
Patches 3-5 refactor and extend the testsuite to validate that patch 2 fixes the issue. Note that these tests fail on master.
Patch 6 attempts to incrementally improve the upgrade / policy docs to make this behaviour more clear.

Patch 2 description for reference:
policy: Treat remote-node as host in resolution

In the --enable-remote-node-identity=false mode, the policy behaviour is
intended to be identical to Cilium v1.6 or earlier, which is to say that
if a remote node sends traffic towards a pod on the local node, then
that traffic should be allowed.

Since commit 28dc941544f1 ("bpf: Map HOST_ID to REMOTE_NODE_ID when
encapsulating"), we have propagated the source identity from remote
nodes as the remote node identity rather than the host, which breaks the
above policy expectations because policy is looked up using this new
remote-node identity.

We could entirely disable the remote-node identity, but we want to
provide a path whereby eventually all users are able to apply separate
policy for whether they allow traffic from the localhost (where the
typical use can is to handle kubelet health checks) vs. other nodes
(where the typical case is hostNetworking pods on other nodes). This may
allow some users to tighten their security posture.

With this approach, the policy handling on the datapath will allow
traffic from remote nodes if the relevant daemon options are configured,
but in visibility tools like cilium monitor or Hubble, the actual
identity used will be propagated. For users to migrate from
`--enable-remote-node-identity=false` to setting this option to true,
they can first deploy with it set to false, then monitor their traffic,
craft their policies, and subsequently perform a migration step to turn
this flag on. This can be iterated in a loop separately from the concern
of upgrading their Cilium version.

Supersedes: https://github.com/cilium/cilium/pull/10951
Fixes: https://github.com/cilium/cilium/issues/10702

test-me-please

test-me-please

test-focus "Multi-node policy"

Coverage increased (+0.02%) to 46.818% when pulling e0381f3aac7639154f844f4432ee63672a153971 on joestringer:submit/fix-remote-node-policy into 4ce5a4ed9a7ceedf4997a3c2ee895b89bfb58695 on cilium:master.

test-me-please

Existing CI runs were all green. Here's my incremental diff, I will run the CI once more.
diff --git a/test/k8sT/Policies.go b/test/k8sT/Policies.go
index 21c8b2847a7e..bb22d5aae6dd 100644
--- a/test/k8sT/Policies.go
+++ b/test/k8sT/Policies.go
@@ -1023,20 +1023,25 @@ var _ = Describe("K8sPolicyTest", func() {
                                _, k8s2PodIP = kubectl.GetPodOnNodeWithOffset(k8s2Name, testDS, 0)
                        })

+                       AfterAll(func() {
+                               By("Redeploying Cilium with default configuration")
+                               RedeployCilium(kubectl, ciliumFilename, daemonCfg)
+                       })
+
                        validateNodeConnectivity := func(expectHostSuccess, expectRemoteNodeSuccess bool) {
-                               By("Checking connectivity from k8s1 to local pod")
+                               By("Checking ingress connectivity from k8s1 node to k8s1 pod (host)")
                                res, err := kubectl.ExecInHostNetNS(context.TODO(), k8s1Name,
                                        helpers.CurlFail(k8s1PodIP))
                                Expect(err).To(BeNil(), "Cannot run curl in host netns")
                                ExpectWithOffset(1, res).To(getMatcher(expectHostSuccess),
                                        "HTTP ingress connectivity to pod %q from local host", k8s1PodIP)

-                               By("Checking connectivity from k8s1 to k8s2 pod")
+                               By("Checking ingress connectivity from k8s1 node to k8s2 pod (remote-node)")
                                res, err = kubectl.ExecInHostNetNS(context.TODO(), k8s1Name,
                                        helpers.CurlFail(k8s2PodIP))
                                Expect(err).To(BeNil(), "Cannot run curl in host netns")
                                ExpectWithOffset(1, res).To(getMatcher(expectRemoteNodeSuccess),
-                                       "HTTP ingress connectivity to pod %q from remote node", k8s1PodIP)
+                                       "HTTP ingress connectivity to pod %q from remote node", k8s2PodIP)
                        }

                        importPolicy := func(file, name string) {
@@ -1058,11 +1063,6 @@ var _ = Describe("K8sPolicyTest", func() {
                                        RedeployCilium(kubectl, ciliumFilename, newCfg)
:...skipping...
diff --git a/pkg/policy/l4.go b/pkg/policy/l4.go
index 070fc902e0e6..b9a919389d0a 100644
--- a/pkg/policy/l4.go
+++ b/pkg/policy/l4.go
@@ -578,8 +578,8 @@ func createL4IngressFilter(policyCtx PolicyContext, fromEndpoints api.EndpointSe
        // then wildcard Host at L7.
        if !rule.Rules.IsEmpty() && len(hostWildcardL7) > 0 {
                for cs := range filter.L7RulesPerSelector {
-                       for _, name := range hostWildcardL7 {
-                               if cs.Selects(identity.ReservedIdentityHost) {
+                       if cs.Selects(identity.ReservedIdentityHost) {
+                               for _, name := range hostWildcardL7 {
                                        selector := api.ReservedEndpointSelectors[name]
                                        filter.cacheIdentitySelector(selector, policyCtx.GetSelectorCache())
                                }
diff --git a/test/k8sT/Policies.go b/test/k8sT/Policies.go
index 21c8b2847a7e..bb22d5aae6dd 100644
--- a/test/k8sT/Policies.go
+++ b/test/k8sT/Policies.go
@@ -1023,20 +1023,25 @@ var _ = Describe("K8sPolicyTest", func() {
                                _, k8s2PodIP = kubectl.GetPodOnNodeWithOffset(k8s2Name, testDS, 0)
                        })

+                       AfterAll(func() {
+                               By("Redeploying Cilium with default configuration")
+                               RedeployCilium(kubectl, ciliumFilename, daemonCfg)
+                       })
+
                        validateNodeConnectivity := func(expectHostSuccess, expectRemoteNodeSuccess bool) {
-                               By("Checking connectivity from k8s1 to local pod")
+                               By("Checking ingress connectivity from k8s1 node to k8s1 pod (host)")
                                res, err := kubectl.ExecInHostNetNS(context.TODO(), k8s1Name,
                                        helpers.CurlFail(k8s1PodIP))
                                Expect(err).To(BeNil(), "Cannot run curl in host netns")
                                ExpectWithOffset(1, res).To(getMatcher(expectHostSuccess),
                                        "HTTP ingress connectivity to pod %q from local host", k8s1PodIP)

-                               By("Checking connectivity from k8s1 to k8s2 pod")
+                               By("Checking ingress connectivity from k8s1 node to k8s2 pod (remote-node)")
                                res, err = kubectl.ExecInHostNetNS(context.TODO(), k8s1Name,
                                        helpers.CurlFail(k8s2PodIP))
                                Expect(err).To(BeNil(), "Cannot run curl in host netns")
                                ExpectWithOffset(1, res).To(getMatcher(expectRemoteNodeSuccess),
-                                       "HTTP ingress connectivity to pod %q from remote node", k8s1PodIP)
+                                       "HTTP ingress connectivity to pod %q from remote node", k8s2PodIP)
                        }

                        importPolicy := func(file, name string) {
@@ -1058,11 +1063,6 @@ var _ = Describe("K8sPolicyTest", func() {
                                        RedeployCilium(kubectl, ciliumFilename, newCfg)
                                })

-                               AfterAll(func() {
-                                       By("Reconfiguring Cilium to enable remote-node identity")
-                                       RedeployCilium(kubectl, ciliumFilename, daemonCfg)
-                               })
-
                                It("Allows from all hosts with cnp fromEntities host policy", func() {
                                        By("Installing fromEntities host policy")
                                        importPolicy(cnpFromEntitiesHost, "from-entities-host")
@@ -1073,6 +1073,17 @@ var _ = Describe("K8sPolicyTest", func() {
                        })

                        Context("with remote-node identity enabled", func() {
+                               BeforeAll(func() {
+                                       By("Reconfiguring Cilium to enable remote-node identity")
+                                       newCfg := map[string]string{
+                                               "global.remoteNodeIdentity": "true",
+                                       }
+                                       for k, v := range daemonCfg {
+                                               newCfg[k] = v
+                                       }
+                                       RedeployCilium(kubectl, ciliumFilename, newCfg)
+                               })
+
                                It("Validates fromEntities remote-node policy", func() {
                                        By("Installing default-deny ingress policy")
                                        importPolicy(cnpDenyIngress, "default-deny-ingress")

test-me-please

docs look good, chatted with joe about this in slack.

Hit https://github.com/cilium/cilium/issues/11024 , but I don't see how it can be related to the incremental changes I did since the last run. Also only affects net-next runs.

test-focus K8sServicesTest Checks service across nodes Tests NodePort BPF Tests with XDP

If the remote-node identity is disabled, they are treated as host and thus allowed. It means that we are actually more permissive. One could argue it is more consistent but it's also a change.

This is a valid concern, let's follow up on this separately. I think one big aspect is how we want things to work longer-term so we can reduce operational complexity, and the other big aspect is that assuming we stay with the current behaviour, how we ensure a smooth transition for existing users.


Tidy up render_modal_workflow

Also, now returns valid JSON with correct content type header

Are you sure this works? In the existing code, the 'onload' item in the dict is a function object - not a string:
{
    'html': '
some html
',
    'onload': function(modal) {...}
}

By converting the whole thing to JSON, you've turned the 'onload' item into a string - which the modal-workflow code is going to attempt (and fail) to call as a function.

Ohh, I see now. Maybe we should look into changing that....


fix: make the url in inline style correct

Bug Reson
Use inline styles to load styles sync, relative paths in styles will be wrong.
Fix Soultion
Use postcss-url to fix relative paths.

Can you provide a reproduction of the bug first?

Oh I get it now... it's a regression caused by f6a21b2

Closed by accidentally referencing the wrong issue.

Thanks a lot for this PR. This is a right fix, but I didn't merge it because we already have internal utils for rewriting relative URLs in CSS which is pretty lightweight and it's best to avoid introducing another dependency / postcss processing step. See e483fc6 for more details.


Encode/Decode Argument and Returned Values

Encode/Decode Argument and Returned Values
Description
Currently, the SDK lacks functionality to encode arguments and decode returned values for methods of type routine or deployers. This limitation restricts developers from seamlessly passing raw arguments and receiving decoded responses.
Proposed Solution
To address this issue, we propose exposing a set of APIs within the SDK that would enable developers to pass raw arguments. The SDK should then be capable of polarizing (encoding) the arguments and depolarizing (decoding) the returned values based on the provided manifest.
This enhancement would streamline the development workflow by allowing developers to interact with the SDK using raw data formats, eliminating the need for manual encoding and decoding steps. The SDK would handle the necessary transformations transparently, ensuring compatibility with the underlying system and improving the overall developer experience.

closed #81


[YouTube] Unable to authenticate with enabled TFA

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.12.31. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.12.31

D:\downloaded_audio
λ youtube-dl --version
2017.12.31

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: ['-x', '--audio-format', 'mp3', '--audio-quality', '0']
[debug] Custom config: []
[debug] Command-line args: ['-n', '-v', 'https://www.youtube.com/watch?v=tM8x5Zhe5O0']
[debug] Encodings: locale cp1251, fs utf-8, out utf-8, pref cp1251
[debug] youtube-dl version 2017.12.31
[debug] Python version 3.6.3 - Windows-10-10.0.16299-SP0
[debug] exe versions: ffmpeg N-88355-g05beee44c6, ffprobe N-88355-g05beee44c6
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]:
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request
WARNING: Unable to extract CheckCookie URL
[youtube] tM8x5Zhe5O0: Downloading webpage
[youtube] tM8x5Zhe5O0: Downloading video info webpage
ERROR: tM8x5Zhe5O0: YouTube said: Please sign in to view this video.
Traceback (most recent call last):
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 784, in extract_info
    ie_result = ie.extract(url)
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\extractor\common.py", line 438, in extract
    ie_result = self._real_extract(url)
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\extractor\youtube.py", line 1609, in _real_extract
    expected=True, video_id=video_id)
youtube_dl.utils.ExtractorError: tM8x5Zhe5O0: YouTube said: Please sign in to view this video.

[debug] System config: []
[debug] User config: ['-x', '--audio-format', 'mp3', '--audio-quality', '0']
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-v', 'https://www.youtube.com/watch?v=tM8x5Zhe5O0']
[debug] Encodings: locale cp1251, fs utf-8, out utf-8, pref cp1251
[debug] youtube-dl version 2017.12.31
[debug] Python version 3.6.3 - Windows-10-10.0.16299-SP0
[debug] exe versions: ffmpeg N-88355-g05beee44c6, ffprobe N-88355-g05beee44c6
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]:
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request
WARNING: Unable to extract CheckCookie URL
[youtube] tM8x5Zhe5O0: Downloading webpage
[youtube] tM8x5Zhe5O0: Downloading video info webpage
ERROR: tM8x5Zhe5O0: YouTube said: Please sign in to view this video.
Traceback (most recent call last):
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 784, in extract_info
    ie_result = ie.extract(url)
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\extractor\common.py", line 438, in extract
    ie_result = self._real_extract(url)
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\extractor\youtube.py", line 1609, in _real_extract
    expected=True, video_id=video_id)
youtube_dl.utils.ExtractorError: tM8x5Zhe5O0: YouTube said: Please sign in to view this video.

[debug] System config: []
[debug] User config: ['-x', '--audio-format', 'mp3', '--audio-quality', '0']
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-2', '123456', '-v', 'https://www.youtube.com/watch?v=tM8x5Zhe5O0']
[debug] Encodings: locale cp1251, fs utf-8, out utf-8, pref cp1251
[debug] youtube-dl version 2017.12.31
[debug] Python version 3.6.3 - Windows-10-10.0.16299-SP0
[debug] exe versions: ffmpeg N-88355-g05beee44c6, ffprobe N-88355-g05beee44c6
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request
WARNING: Unable to extract CheckCookie URL
[youtube] tM8x5Zhe5O0: Downloading webpage
[youtube] tM8x5Zhe5O0: Downloading video info webpage
ERROR: tM8x5Zhe5O0: YouTube said: Please sign in to view this video.
Traceback (most recent call last):
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 784, in extract_info
    ie_result = ie.extract(url)
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\extractor\common.py", line 438, in extract
    ie_result = self._real_extract(url)
  File "d:\program files\python36-32\lib\site-packages\youtube_dl\extractor\youtube.py", line 1609, in _real_extract
    expected=True, video_id=video_id)
youtube_dl.utils.ExtractorError: tM8x5Zhe5O0: YouTube said: Please sign in to view this video.

Description of your issue, suggested solution and other information
Unable to authenticate with enabled TFA on account.
Returns HTTP Error 400 after typing 2-step verification code.
Happens in any way of authentication: .netrc, direct passing login+pass via cmd args, direct passing login+pass+tfacode via cmd args.

Same *it here. Looks like cookie was expired and app tried to login back, but can't get thru TFA
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-s', u'--verbose', u':ytwatchlater', u'--ignore-config', u'-u', u'PRIVATE', u'--dump-pages']
Type account password and press [Return]:
[debug] Encodings: locale UTF-8, fs utf-8, out None, pref UTF-8
[debug] youtube-dl version 2018.05.18
[debug] Python version 2.7.15 (CPython) - Darwin-17.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 4.0, ffprobe 4.0, rtmpdump 2.4
[debug] Proxy map: {}
Type 2-step verification code and press [Return]:
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
ERROR: This playlist does not exist.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 503, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2776, in _real_extract
    _, playlist = self._extract_playlist('WL')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2291, in _extract_playlist
    raise ExtractorError(message, expected=True)
ExtractorError: This playlist does not exist.

Just for sure with Python 3.x.x
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-s', '--verbose', ':ytwatchlater', '--ignore-config', '-u', 'PRIVATE', '--dump-pages']
Type account password and press [Return]:
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.05.18
[debug] Python version 3.6.5 (CPython) - Darwin-17.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 4.0, ffprobe 4.0, rtmpdump 2.4
[debug] Proxy map: {}
Type 2-step verification code and press [Return]:
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
ERROR: This playlist does not exist.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 503, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2776, in _real_extract
    _, playlist = self._extract_playlist('WL')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2291, in _extract_playlist
    raise ExtractorError(message, expected=True)
youtube_dl.utils.ExtractorError: This playlist does not exist.

--dump-pages here

Receiving same error with Python 2.7.x
[debug] System config: []
[debug] User config: [u'-x', u'--audio-format', u'm4a', u'-o', u'~/Desktop/%(playlist_title)s/%(title)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'https://www.youtube.com/playlist?list=WL']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.06.04
[debug] Python version 2.7.10 (CPython) - Darwin-17.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.3.3, ffprobe 3.3.3, phantomjs 2.1.1
[debug] Proxy map: {}
[youtube:watchlater] Downloading login page
[youtube:watchlater] Looking up account info
[youtube:watchlater] Logging in
Type 2-step verification code and press [Return]:
[youtube:watchlater] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
[youtube:watchlater] WL: Downloading webpage
ERROR: This playlist does not exist.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 500, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2790, in _real_extract
    _, playlist = self._extract_playlist('WL')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2305, in _extract_playlist
    raise ExtractorError(message, expected=True)
ExtractorError: This playlist does not exist.

@Vulp35 I'm working on a fix. What 2FA methods (Authenticator code, texted code, prompt in Google/Gmail app) do you have turned on on your Google account? Which one is the default, the one it asks for when you try to log in with a browser (use incognito if you don't know)?

@eduardog3000 , for mine, my YouTube 2FA is via Google Authenticator.
Enter a verification code
Get a verification code from the Google Authenticator app
Enter the 6-digit code

@sandalle What error do you get when you try to log in with youtube-dl? HTTP Error 500: Internal Server Error or HTTP Error 400: Bad Request.
Then can you download this branch and tell me what happens when you try to log in.

@eduardog3000
$ youtube-dl --version
2018.12.09

$ youtube-dl -u<EMAIL_ADDRESS>-2 https://www.youtube.com/watch?v=UcRZQL-grSs
Type account password and press [Return]:
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: You must provide at least one URL.
Type youtube-dl --help to see a list of all options.

$ youtube-dl -u<EMAIL_ADDRESS>-p -2 https://www.youtube.com/watch?v=UcRZQL-grSs
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
WARNING: Unable to login: Invalid password
[youtube] UcRZQL-grSs: Downloading webpage
[youtube] UcRZQL-grSs: Downloading video info webpage
^C
ERROR: Interrupted by user

I'm unsure why those aren't working, but using my ~/.netrc values worked in that I got HTTP Error 500 for 2FA:
$ youtube-dl -n https://www.youtube.com/watch?v=UcRZQL-grSs
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]:
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
[youtube] UcRZQL-grSs: Downloading webpage
[youtube] UcRZQL-grSs: Downloading video info webpage
[download] Destination: The Diver's Grave _ Critical Role _ Campaign 2, Episode 44-UcRZQL-grSs.mp4
[download]   0.1% of 920.30MiB at 814.70KiB/s ETA 19:16^C
ERROR: Interrupted by user

I've cloned git@github.com:eduardog3000/youtube-dl.git seems to work with 2FA. :)
$ git checkout -b tfa-fix origin/tfa-fix
Branch 'tfa-fix' set up to track remote branch 'tfa-fix' from 'origin'.
Switched to a new branch 'tfa-fix'

$ python -m youtube_dl --version
2018.12.09

$ python -m youtube_dl -v -n https://www.youtube.com/watch?v=UcRZQL-grSs
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-n', u'https://www.youtube.com/watch?v=UcRZQL-grSs']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.12.09
[debug] Git HEAD: 40217df7e
[debug] Python version 2.7.15 (CPython) - Darwin-18.2.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]:
[youtube] Selecting TOTP TFA challenge
[youtube] Submitting TFA code
[youtube] Checking cookie
[youtube] UcRZQL-grSs: Downloading webpage
[youtube] UcRZQL-grSs: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on u'https://r5---sn-nx5s7n7s.googlevideo.com/videoplayback?signature=2A320C3FB21974C9F1211145558B3196BB5E8096.58844199BEE09D65408089751A52812BECE6CD43&expire=1544763840&source=youtube&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&txp=5535432&ip=2600%3A1%3A9810%3Aad5b%3A9095%3A43ad%3A377%3A83b6&fvip=5&itag=22&mime=video%2Fmp4&id=o-ANcaqnqgoVSCJDZq4rs6YxKGWaPKe3eJbZLQ6xvDsDEY&ratebypass=yes&requiressl=yes&mn=sn-nx5s7n7s%2Csn-nx57ynlz&mm=31%2C29&c=WEB&ipbits=0&initcwndbps=617500&key=yt6&lmt=1544553774170529&dur=16280.555&mv=m&pl=44&mt=1544742111&ms=au%2Crdu&ei=YOUSXN6ZBcrHkgbGlqCgAQ'
[download] Resuming download at byte 6225101
[download] Destination: The Diver's Grave _ Critical Role _ Campaign 2, Episode 44-UcRZQL-grSs.mp4
[download]   1.1% of 920.30MiB at 275.75KiB/s ETA 56:21^C
ERROR: Interrupted by user

I also tried downloading a playlist, and the first item worked with 2FA and the second continued to work with the 2FA session. :)
[youtube] Checking cookie
[youtube] YW01AC2NJAc: Downloading webpage
[youtube] YW01AC2NJAc: Downloading video info webpage
[youtube] YW01AC2NJAc: Marking watched

If I were you I'd mask your logins.

@sandalle
$ youtube-dl -u<EMAIL_ADDRESS>-p -2 https://www.youtube.com/watch?v=UcRZQL-grSs
is because you have the syntax wrong, you have to put your 6 digit TOTP code after -2.
But it's good to see my code works for you. Just to be sure, this is the very first screen you see after inputting your password in a normal browser?

Not something like this:

@sandalle
But it's good to see my code works for you. Just to be sure, this is the very first screen you see after inputting your password in a normal browser?

Correct, that's what I see after using a browser in Privacy mode.

@sandalle Thank you. Now I need someone who gets the HTTP Error 400: Bad Request (and probably should have the app prompt as their default method) to make sure I fixed that too.
Once I confirm that I should be ready to submit a PR.

@Vulp35 I'm working on a fix. What 2FA methods (Authenticator code, texted code, prompt in Google/Gmail app) do you have turned on on your Google account? Which one is the default, the one it asks for when you try to log in with a browser (use incognito if you don't know)?

I can add prompt aswell if that helps you.

Oh, good, Prompt started working for me (you can see in the image above I got locked out of it), so I could test it myself.
Looks like all is good, I'll submit the PR.

Oh, good, Prompt started working for me (you can see in my image above I got locked out of it), so I could test it myself.

Anything special I have to do for prompt? I accepted then hit enter but that didn't seem to work?

@Vulp35 youtube-dl can't actually use the prompt, it still needs the Google Authenticator code, but I wanted to make sure that someone with the prompt enabled was still able to use the code through youtube-dl. The way the fix works I wasn't completely sure it would work when Google asks for the prompt by default.

@eduardog3000 oh gotcha! Yup can also confirm it working with prompt enabled as the default but using a code from Authenticator in youtube-dl

I am having the same difficulty with the version 2018.11.23 and the forked version that was modified by @eduardog3000 . This was difficult to test on Windows without an executable, but I figured it out. In any case, when I temporarily disable 2FA on my account, both the stock and modified versions worked for me. So I am not sure what the fix was supposed to do, but I saw no change in behavior.
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
WARNING: Unable to log in: HTTP Error 400: Bad Request

I'm still having this issue in version 2019.02.08. I use SMS messages as my TFA method.
Version proof:
.\youtube-dl.exe --version
2019.02.08

Debug output on Windows:
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [':ytwatchlater', '-i', '--get-id', '-v', '-r', '4.2M', '--username', 'PRIVATE', '-p', 'PRIVATE']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.02.08
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17134
[debug] exe versions: ffmpeg N-80386-g5f5a97d, ffprobe N-80386-g5f5a97d
[debug] Proxy map: {}
Type 2-step verification code and press [Return]:
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request
ERROR: This playlist does not exist.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp6w61832c\build\youtube_dl\YoutubeDL.py", line 794, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp6w61832c\build\youtube_dl\extractor\common.py", line 508, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp6w61832c\build\youtube_dl\extractor\youtube.py", line 2923, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp6w61832c\build\youtube_dl\extractor\youtube.py", line 2435, in _extract_playlist
youtube_dl.utils.ExtractorError: This playlist does not exist.

Side note:
First, I was unable to log in using an App Password, which should bypass TFA. This comment  on #5280 led me to attempt to login using my regular password and TFA instead (not my preference, but I was willing to try it as a workaround). But that's failing too, which led me to this ticket.
It seems like both App Passwords and TFA are broken in version 2019.02.08. Can someone familiar with how to correctly use youtube-dl with an App Password confirm/debunk that? If both are broken, that makes it very difficult to get private playlists of any kind.

@uruiamme Which version is that log from? My fork might be broken for reasons other than TFA because it is a fork of an old version. The newest official version still doesn't work with TFA because no fix has been merged in.
This branch is both the newest youtube-dl version, and has my fix applied. It should work for you and anyone who has TOTP TFA enabled. If not, post the log from it.

@eduardog3000 I had followed your link to https://github.com/eduardog3000/youtube-dl/tree/tfa-fix
but you just gave this new link:
https://github.com/eduardog3000/youtube-dl/commits/tfa-fix-2019
So my test found out that both 2018.11.23 (stock) and yours based on 2018.11.23 were both still not working. I have not tried the new one. Why would the new one work? Did your fix change anything or did the main trunk make changes to TFA problems?

@uruiamme 2018.11.23 stock is old and doesn't work anymore for reasons unrelated to TFA. YouTube changes things a lot, and youtube-dl has to keep up. My version based on 2018.11.23 doesn't work for the same reason.
The new branch is based on the latest commit to this main repo, so it should work where 2018.11.23 doesn't. But if YouTube has changed anything about TFA since then, my fix might not work no matter the base version.
Try my tfa-fix-2019 branch, if it doesn't work, paste the error log.

@dstftw @remitamine
I apologize for possibly tagging excessively, but 2FA on Youtube has been broken for months, yet a working fix has existed for almost as long.
Can you at least say what the current blockers are for merging the fix? The latest PR was simply locked with no further feedback given.
Links:

The original PR fixing this issue
A new branch by the original author of the PR with the same fix but based on a more recent commit of the master branch

@eduardog3000 just curious, why create a new branch instead of rebasing the old one?

@FranciscoPombal as a quick fix for the person who was asking me about it here. If the maintainers are willing to merge my code I'd happily get it all sorted. Although the code in my PR, while based on an old upstream commit, can still be merged without conflict.

Is there some lingering issue with the PR that's preventing merge?

@LoneFenris There was an issue, I fixed it, then the collaborator never responded and eventually locked the conversation.

Still having this issue with youtube-dl version 2019.10.29

@raffaem From the conversation on the associated PR, it appears there may be two issues with it:

There is a concern that having TFA support could cause ytdl to overflow the number of TFA challenges Google allows in a time period

There is a suggested improvement to how the PR responds to challenges that is yet to be implemented


fix: redirection to Home page after successful registration.

Pull Request
What does this PR does?
This pull request implements the use of the useNavigate hook from React Router to redirect the user to the home page after a successful registration. Previously, there was no automatic redirection, and the user had to manually navigate to the home page. Now, upon successful form submission, the user will be directed to the home page automatically.

Related Issue
Fixes #282

Screen Recording
https://github.com/user-attachments/assets/d2417a2a-ceb4-484d-a9b7-030be456f948

Hey @angadnagar please resolve the merge conflicts.

@angadnagar resolve merge conflicts.

@gauravsingh1281 all conflicts are merged you can check.


Add a '.' to the help text for consistency

Was about to submit a full patch for the unescaped %'s but @sbeaver-netgate beat me to it by a couple of minutes. So here's my minuscule contribution!

 Then we will all be working off the same page, rather than having to remember which bits have already been fixed in pending PRs.


support qwen2 inferece model

PR types

Others
PR changes

Others
Description

suppoet qwen2 inferece model

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: zhoutianzi666:x: Sunny-bot1You have signed the CLA already but the status is still pending? Let us recheck it.


[Question] Private Calendar Blockchain

Hi Chainpoint crew,
Is there a way to run a private Calendar blockchain on premise by running some Chainpoint nodes on our servers and anchor the merkle root in a public Blockchain ?
Or do we have to fork the code ?
Thanks !

You currently can run a private chainpoint node and that will send merkle roots to chainpoint core for timestamping against the chainpoint calendar and the Bitcoin blockchain.  The chainpoint calendar is utilized for the entire chainpoint network without leaking info or data to other participants.
May I ask why you would want to run your own "chainpoint calendar blockchain" instance?

Hi. Thank you :)
Me need is : run my own (private) chainpoint core (with multiple chainpoint node) on premise on our servers. Without connecting to the public chainpoint core.
Is that possible ?
I want to run my own chainpoint core because we want to be sure that the core will never stop running

@mcanzerini We don't have documentation on how-to easily run chainpoint core.
We are currently in the process of making core more decentralized and easier to run. Stayed tuned to our social media / telegram / twitter / discord / github.
We are trying to make the most robust time-stamping service and hope you will want to be part of the main chainpoint public network. This includes individuals and groups having the opportunity to help run chainpoint core.

As I mentioned we are in the process of changing up core, but yes currently all of our core software is open source:
https://github.com/chainpoint/chainpoint-services/blob/develop/README.md


Adhere to branch naming convention

The plugin should propose a name for a branch. Depending on the related work item. Or at least based on the intended use of the branch.
A commonly accepted naming convention is the one defined by GitFlow (ie release/...., feature/... and hotfix/...).

When creating branch a default name is selected workitem0000
It will be amazing if the user can specify the default to be userdefault0000


[WIP] chore: wrap in resizable only in dev

Make sure we wrap within a resizable, which is used purely for dev purposes, only when the filter-pane is built in development mode.
This should hopefully help a problem we seen in rendering tests where the chart didn't get the correct size.

Update snapshot action seem to work :)


To Have Different Address Used For Ethereum and ERC20 Tokens Or Same Address?

They are different now. You should be sure you have required money on ETH ERC20 address to make transactions.


Finding A requests that have been resolved via CNAME

I don't know if this is a limitation of pycares, or if I'm missing something obvious.
Pycares seems to automatically follow CNAME entries to resolve the hostname to an IP address, but gives no indication of when it has done so. Is there any way to tell when this has happened, and if so what the CNAME was?
Thanks very much.

An example for  you:
 $ dig photos.google.ca

; <<>> DiG 9.14.2 <<>> photos.google.ca
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41107
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;photos.google.ca.              IN      A

;; ANSWER SECTION:
photos.google.ca.       10785   IN      CNAME   picasaweb.google.com.
picasaweb.google.com.   1580    IN      CNAME   picasaweb.l.google.com.
picasaweb.l.google.com. 900     IN      A       <IP_ADDRESS>

Will only return something like [<ares_query_a_result> host=<IP_ADDRESS>, ttl=62] with no indication that it was resolved via CNAME.

Digging in to this further, it looks like gethostbyname kinda provides the information I am looking for.
The information returned (using the pycares asyncio example) is as follows when doing this request: resolver.gethostbyname('photos.google.ca', cb)
Result: <ares_host_result> name=picasaweb.l.google.com, aliases=['photos.google.ca', 'picasaweb.google.com'], addresses=['<IP_ADDRESS>'], Error: None

So while it doesn't directly indicate that a CNAME request was resolved, the presence of a name other than the original name requested along with the aliases list seems like enough indication that it happened. I have yet to inspect the aiodns source but I assume you're returning all the information from pycares...
One suggestion, perhaps you should make a note in the README.md file to this difference in the return values provided by the function? Could be helpful to anyone looking in the future!

Hi there!
I think this could be tackled if we pass an array of hostent entries here: https://github.com/saghul/pycares/blob/master/src/pycares/init.py#L150
That way we can get access to the aliases, like gethostbyname does. I'll try to take a look, but feel free to experiment and send a PR if you find it works!

I do miss this functionality very much. In my case, I want to know all (recursive) CNAME records even if they don't resolve to an IP. Dig gives all this information when doing A query. In this case even gethostbyname does not help - it only seems to return CNAME records when the domain actually resolves to an IP. The only way I can think of getting this information using aiodns right now if by manually doing recursive CNAME queries, which seems unnecessarily complex.

Last I checked that information is meeting in the c-ares structures so there is not much that we can do other than faking it by recursing if necessary in aiodns itself.
Parches to improve this are most welcome!


PLNSRVCE-1310: bump pipeline service in dev/staging, bump prod to prior staging version

bumping dev and staging from Revert Upgrade PaC version to 0.17.3 on June 8 to bump exporter mem to 512 Mib on June 13; also includes the removal of chains-ca-cert; the other changes were related to CI in the pipeline-service repo
bumping prod to our prior level for staging that has been vetted, which is the Revert Upgrade PaC version to 0.17.3 on June 8; this will bring in the reset of the exporter changes to date, including the new labeling and the switch to an image fully produced by the latest RHTAP build-definitions pipeline

Once CI and staging is happy with  12862f674f40f65cf94d48691e3fe1a2d3bf95a6 I will bump prod to that level.

saw some unrelated flakes in cluster setup tailing logs live ... going ahead nd rebasing / restarting

another cluster setup / hypershift interaction looking failure ..... may need to engage in qe forum seeing a lot of red in https://prow.ci.openshift.org/job-history/gs/origin-ci-test/pr-logs/directory/pull-ci-redhat-appstudio-infra-deployments-main-appstudio-e2e-tests
/retest

/lgtm


Incorrect content

File: network/network-tutorial-overlay.md, CC @gbarr01
Communicate between a container and a swarm service，this section does not seem to match the title

As far as I can tell the section is correct. The problem seems to be that that when you click the link, the site ends up scrolling a bit more than what you expect it to. That problem is being tracked at #2199, so I'll close this one as a duplicate.

that section explains the communication between 2 containers in default bridge network, not "Communicate between a container and a swarm service". So its not correct and needs to be fixed.


Use binary search in SkiaParagraph.lineMetricsForOffset

Proposed Changes
Use binary search in SkiaParagraph.lineMetricsForOffset. Currently the search is linear.
Testing
Test: Ran the user-provided reproducer. Also tested manually that caret is placed correctly when clicking in a multi-line textfield.
Issues Fixed
Fixes: https://github.com/JetBrains/compose-multiplatform/issues/4021

I changed List<T>.fastBinarySearch from commonMain to be internal (was private before), to avoid reimplementing binary search.


Commentary API Backend

This PR implements the commentary API.
Ended up having separate calls for creating initial comments (only supports versioned document links now), from creating replies. The reason for that is to make it less error-prone as to what target should be used in replies. Especially because the semantics of this regarding to updating comments are unclear. This way when we introduce updates, the frontend wouldn't need to change this part probably.
I've been researching various APIs that do comments, like YouTube, Google Docs, Notion, etc. And some of them have this distinction between replies and comments, and I figured it could be helpful.

Work continues in feat/commentary branch.