Use BotFramework-WebChat in a React app with customization

I have a question
Hello,
I want to build a Chat bot in a custom React application (public web site).
I have a few questions:

Can we have an interface like in https://www.intercom.com?. An icon appears in the bottom corner of the web site and if you click, you open the chatbot interface.
Is it possible to fully customize the web chat UI?
Is it possible to change the code of the chatbot in our application?
Is it possible to change the code of the web chat in order to call our APIs that will receive a prompt and generate a response?
Is it possible to change the Html and CSS?

If this is possible, can you provide links to documentation on how to to this?
Thanks

if you post this as a question in Stack Overflow, I will be more than happy to answer it.

Hello @HesselWellema,
Please respond at https://stackoverflow.com/questions/78678224/use-botframework-webchat-in-a-react-app-with-customization.
Thanks

@miguelisidoro - Please find the responses to your questions, below.

Can we have an interface like in intercom.com?. An icon appears in the bottom corner of the web site and if you click, you open the chatbot interface.

A sample demonstrating this already exists in the samples folder of this repo, titled 06.recomposing-ui/a.minimizable-web-chat. Please review this to see if it meets your needs.

Is it possible to fully customize the web chat UI?

Web Chat is not fully customizable, though it is highly customizable. If you look over the various samples you will find that you can affect Web Chat in a variety of ways including building custom components and recomposing the UI, implementing different built-in style options, and altering functionality. However, Web Chat is built using React which handles updates to the DOM. This means making UI changes outside of Web Chat is not recommended as a best practice.

Is it possible to change the code of the chatbot in our application?

Web Chat is only a client application allowing the user to communicate with the bot. Changes made to the code of your bot are separate from Web Chat. Changes made in one should have no bearing on the other outside of conversational flow.

Is it possible to change the code of the web chat in order to call our APIs that will receive a prompt and generate a response?

Web Chat runs in the browser and consists of HTML and JavaScript. Any expected functionality that exists in a normal web app can be implemented in Web Chat. For instance, suppose a bot presents the user with several options of which one option, when selected, sends data to an API. The user, selecting that option, could trigger an event in the page that then posts the data to the API via an event listener. (There is a basic sample demonstrating this concept, too!)

Is it possible to change the Html and CSS?

As mentioned above, yes, but with limitations. The samples in 02.branding-styling-and-customization demonstrate how this can be achieved in most instances.

Thank you @stevkan, I had those doubts too. My project is very similar to miguelisidoro. I ended up creating the bot inside an HTML wrapper where I define a custom header:
Render ReactWebChat with useMemo to avoid mutiple calls to api:

Componente template:

Hello @HesselWellema,
Please respond at https://stackoverflow.com/questions/78678224/use-botframework-webchat-in-a-react-app-with-customization.
Thanks
Stevkans answer is correct

@miguelisidoro - Please find the responses to your questions, below.

Can we have an interface like in intercom.com?. An icon appears in the bottom corner of the web site and if you click, you open the chatbot interface.

A sample demonstrating this already exists in the samples folder of this repo, titled 06.recomposing-ui/a.minimizable-web-chat. Please review this to see if it meets your needs.

Is it possible to fully customize the web chat UI?

Web Chat is not fully customizable, though it is highly customizable. If you look over the various samples you will find that you can affect Web Chat in a variety of ways including building custom components and recomposing the UI, implementing different built-in style options, and altering functionality. However, Web Chat is built using React which handles updates to the DOM. This means making UI changes outside of Web Chat is not recommended as a best practice.

Is it possible to change the code of the chatbot in our application?

Web Chat is only a client application allowing the user to communicate with the bot. Changes made to the code of your bot are separate from Web Chat. Changes made in one should have no bearing on the other outside of conversational flow.

Is it possible to change the code of the web chat in order to call our APIs that will receive a prompt and generate a response?

Web Chat runs in the browser and consists of HTML and JavaScript. Any expected functionality that exists in a normal web app can be implemented in Web Chat. For instance, suppose a bot presents the user with several options of which one option, when selected, sends data to an API. The user, selecting that option, could trigger an event in the page that then posts the data to the API via an event listener. (There is a basic sample demonstrating this concept, too!)

Is it possible to change the Html and CSS?

As mentioned above, yes, but with limitations. The samples in 02.branding-styling-and-customization demonstrate how this can be achieved in most instances.

@miguelisidoro - Please find the responses to your questions, below.

Can we have an interface like in intercom.com?. An icon appears in the bottom corner of the web site and if you click, you open the chatbot interface.

A sample demonstrating this already exists in the samples folder of this repo, titled 06.recomposing-ui/a.minimizable-web-chat. Please review this to see if it meets your needs.

Is it possible to fully customize the web chat UI?

Web Chat is not fully customizable, though it is highly customizable. If you look over the various samples you will find that you can affect Web Chat in a variety of ways including building custom components and recomposing the UI, implementing different built-in style options, and altering functionality. However, Web Chat is built using React which handles updates to the DOM. This means making UI changes outside of Web Chat is not recommended as a best practice.

Is it possible to change the code of the chatbot in our application?

Web Chat is only a client application allowing the user to communicate with the bot. Changes made to the code of your bot are separate from Web Chat. Changes made in one should have no bearing on the other outside of conversational flow.

Is it possible to change the code of the web chat in order to call our APIs that will receive a prompt and generate a response?

Web Chat runs in the browser and consists of HTML and JavaScript. Any expected functionality that exists in a normal web app can be implemented in Web Chat. For instance, suppose a bot presents the user with several options of which one option, when selected, sends data to an API. The user, selecting that option, could trigger an event in the page that then posts the data to the API via an event listener. (There is a basic sample demonstrating this concept, too!)

Is it possible to change the Html and CSS?

As mentioned above, yes, but with limitations. The samples in 02.branding-styling-and-customization demonstrate how this can be achieved in most instances.

Hello @stevkan ,
Thanks for detailed response. I have a couple of more questions:

Why is Web Client implemented in JavaScript and not TypeScript?
Regarding your answer "Web Chat is only a client application allowing the user to communicate with the bot. Changes made to the code of your bot are separate from Web Chat. Changes made in one should have no bearing on the other outside of conversational flow.", what do you consider the bot? You consider Web Chat the UI. Is the bot our custom APIs? If yes, is there any documentation on how the API should be called and what the API should respond in the answer?
If my understanding of 2) is not correct, can you please describe the roles of Web Client (I believe the UI), bot and APIs?

Thanks

@miguelisidoro - As before, please find the responses to your questions below.

Why is Web Client implemented in JavaScript and not TypeScript? Why are the React components not JSX files?

Regarding JS vs TS, you can reference this GitHub post which attempts to answer that question. If you are needing additional clarification or have other questions around design/development decisions, such as the use of JSX files, those would best be answered by the author of Web Chat (and the aforementioned post), @compulim.

what do you consider the bot?

In short, the bot is separate code which, in this case, is written using one of the BotBuilder SDKs (dotnet, JS, or Python). This bot code contains the logic that controls and directs the conversational flow when interacting with a user via a connected client, such as Web Chat, Facebook, Slack, Teams, etc.
For more detailed information, I would suggest reading the various articles in the Bot Framework SDK documentation, starting with How bots work - Basics of the Microsoft Bot Framework.

You consider Web Chat the UI. Is the bot our custom APIs? If yes, is there any documentation on how the API should be called and what the API should respond in the answer (swagger or even documentation) for Request and Response?

No, the bot is not your custom APIs. To be clear, the Web Chat client is designed for, and meant to connect to, a bot built with one of the above SDKs. That being said...is it possible to call an API from an event triggered in the browser and inject any API responses into Web Chat's transcript window?

Technically, yes, but not as the primary or only source for conversational responses. However, API responses generated in the browser are often sent to the bot, via Web Chat, to be processed and displayed back to the user via the conversational flow.
Or, the API call is not handled in the browser, at all, and is instead handled by the bot in the bot's code as part of the conversation.

If your goal is to use Web Chat only with API generated responses, I would recommend you instead identify another client for this purpose. By using Web Chat in this way, certain features would be unavailable as they are specific to receiving activities (messages) from a BotBuilder bot. Additionally, future changes to Web Chat could negatively impact your setup since using Web Chat in that way would not be a factor.

If my understanding of 2) is not correct, can you please describe the roles of Web Client (I believe the UI), bot and APIs?
Are the API calls always done as in this sample? Our goal is that the click button triggers a call to an API and then render the answer in the response area.

See my response posted above.

How do we integrate the web chat in our application? Is there any npm package we can install? The option, could not be fork the repo and integrate the relevant code in our application...

Please read the Web Chat Overview article, and associated articles, found in the Bot Framework SDK Documentation. There is additional information in the BotFramework-WebChat repo's README.md and docs folder.
As for forking the repo, it is an open source project. You are welcome to fork the repo and make any changes you see fit so that it serves your needs, including integrating into your application. Just be aware that the BotFramework-WebChat library has a fairly large code base that is tightly integrated with support for activities (messages) received from a BotBuilder bot. Trying to integrate only the relevant code may prove difficult. Also, support provided in this repo is only for the BotFramework-WebChat project, as it is. Support is not provided for endeavors that significantly alter Web Chat from its current design, such as redesigning it to support other bot products or integrating portions of code into other projects.

what do you consider changes outside the WebChat? Having html and CSS of our own outside of Web Chat? I want to integrate Web Chat in a public web site and you I assume are saying that the only customization is to create custom components, styles (that according to your link are limited), correct?

The "changes outside of WebChat" comment is specific to any implementation of Web Chat. In other words, if you have Web Chat integrated into your web site which also contains many other elements, it is only the Web Chat portion that you would want to refrain from making any changes to the DOM directly. How you manage the rest of the web site is not a concern of Web Chat and is completely at your discretion. In fact, whether you want to make changes directly to Web Chat's DOM is entirely up to you. The most I can say is it is not a best practice for the reasons noted in my previous post.

Closing due to inactivity. Please feel free to reopen if the issue persists or you have new details to share.


Use Poetry 1.3 with [poetry] extra

I believe the only thing blocking this is updating
-cleo = {version = "^1.0.0a5", optional = true, allow-prereleases = true}
+cleo = {version = "^2.0.0", optional = true, allow-prereleases = true}
-poetry = {version = "^1.2.0", optional = true}
+poetry = {version = "^1.3.0", optional = true}

Default ranges created by poetry when adding deps are almost always too restrictive for libraries like this plugin. It mostly only has applications in mind for default behavior IMO.
Maybe a more open range would make sense? Otherwise library users will always have conflicts if they have other dependencies. See also https://github.com/enpaul/tox-poetry-installer/issues/82 which is a similar problem although that's a major release so makes a bit more sense.

Hi @brandon-leapyear sorry for the delay in response here. This is a good thing flag, I'll work on getting this out soon.
@nejch The restrictive dependency specification is one of the driving reasons behind #2 . Unfortunately that issue is currently blocked right now, but once done this plugin will drop a dependency on Poetry entirely.

Thanks for looking into this @enpaul! Since https://github.com/enpaul/tox-poetry-installer/issues/82 will probably stay blocked for a while, does it make sense to release this on pypi so people can use poetry 1.3 for the time being?


Process limit across 2 machines not being followed

This might be related to https://github.com/brainopia/sidekiq-limit_fetch/issues/56
Again we experience some weird issue with our sidekiq configuration. We currently have 2 machines which have the same sidekiq configuration:
---
:verbose: true
:concurrency: 10
:tag: sidekiq_disk
:queues:
- - videos
  - 7
:process_limits:
  videos: 1

Whenever we queue jobs in sidekiq it seems that one machine sticks to the limit of only processing one at the time but the other machine is not. No clue what is going on here and we have a deploy script which provisions the machine equally.

When asking the queue programmatically what is is doing we get the following output:
Machine 1:
"Current sidekiq process: e1cffa43-3334-4888-8153-4426dcc2c3b5\n\n  All processes:\nd50199d2-6039-4c98-b7ea-f9a9583a7a54\n099f32b2-aded-4b6f-b5dc-f4ebfde0acbb\na3a54913-293e-456c-9a33-76e03777e9cd\nda0d15d7-1724-4406-ace3-d0b8ad016cc0\nf0dca400-0cde-498a-a30e-13597d815e65\n\n  Stale processes:\n\n\n  Locked queue processes:\n099f32b2-aded-4b6f-b5dc-f4ebfde0acbb\n\n  Busy queue processes:\n099f32b2-aded-4b6f-b5dc-f4ebfde0acbb\n\n  Limit:\nnil\n\n  Process limit:\n1\n\n  Blocking:\n\n"

Machine 2:
 "Current sidekiq process: c7259ef0-99eb-4688-8d2f-4c244b7100b8\n\n  All processes:\nd50199d2-6039-4c98-b7ea-f9a9583a7a54\n099f32b2-aded-4b6f-b5dc-f4ebfde0acbb\na3a54913-293e-456c-9a33-76e03777e9cd\nda0d15d7-1724-4406-ace3-d0b8ad016cc0\nf0dca400-0cde-498a-a30e-13597d815e65\n\n  Stale processes:\n\n\n  Locked queue processes:\n099f32b2-aded-4b6f-b5dc-f4ebfde0acbb\n56470ac6-6909-4c20-8512-de1aaa3f780f\n\n  Busy queue processes:\n099f32b2-aded-4b6f-b5dc-f4ebfde0acbb\n\n  Limit:\nnil\n\n  Process limit:\n1\n\n  Blocking:\n\n"

No clue why this is happening ...

These are our sidekiq deps:

Have the same or similar issue:
my sidekiq.yml
production:
  :concurrency: 15
:queues:
  - ["memory_intensive_job", 1]
  - ["job2", 1]
  - ["job3", 1]
:process_limits:
  memory_intensive_job: 1
  job2: 10
  job3: 10

10 sidekiq worker machines 1 server for populating the queues with 1 redis server
each worker machine has 3 processes each with 15 concurrency as per sidekiq.yml
memory_intensive_job is consumes a lot of memory so limit per process is set to 1

Expectation:
10 machines * 3 processes * 1 limit for memory_intensive_job = 30 concurrent jobs being executed
Reality:
starts with 30 but over time process takes more and more memory_intensive_jobs which eventually ends up in worker server crashing and not being responsive.

Hi,
We also had such concerns.
We no longer have them without knowing why.
But. You can take a look on this issue : https://github.com/brainopia/sidekiq-limit_fetch/issues/84
I wrote a demo to see if there's a concern with process_limits.
Unfortunately (or not), I found nothing. You may be luckier. In any case this can give you a basis for testing a minimal case.

We had a very similar issue.  Multiple machines reading off of the same queue but each machine needed to be limited to processing one job at a time from that queue.
The way we read the documentation it seemed that we needed to use process limits
:process_limits:
  single_resoruce_queue: 1

but I am now thinking this only applies if you are running multiple sidekiq processes on the same machine.
We added
:limits:
  single_resoruce_queue: 1

and are seeing drastically reduced collisions/issues.
Now multiple machines are pulling off the same queue and each machine is only processing 1 job at a time. (of course we have many other queues that use concurrency)

Hey, thanks for your issue! I've just taken over this project/gem and am going through the PRs and issues. I see this issue is stale so I'm going to close it but if you are still experiencing issues, please feel free to re-open.


typo error in unitframes/cfg.lua

I think i found a typo error in section --Threat/DebuffHighlight in gempUI/modules/unitframes/cfg.lua
partytaget = false,
I'm not a programmer but i think it should be
partytarget = false,
Here is a screenshot.

Keep up the good work!
MrD33K

That's actually the same in the original oUF_Skaarj. Maybe It's intended I'll look out for it later.


How to configure rustfmt not breakdown long lines?

How to configure rustfmt to not do this and keep this line as was before?

Set fn_call_width to a large value in your rustfmt.toml


Integrate CST: Work in progress

Continuing CST Work In Progress pull request. Want to keep you informed about the current integration I'm working on.
In this pull request:

Use CST instead of AST with token hacks (https://github.com/mdevils/cst)
Simplify JsFile and rules.
Improve assertions in tests using chai and chai extensions (it is very hard to make huge changes with the current error messages).

Feel free to discuss changes. Any help is very appreciated.
I try to keep everything rebased.
/cc @markelog @hzoo @mrjoelkemp @mikesherov @zxqfox

Since changes are pretty big and will get bigger, i think we need to make this green for the travis - add all those skip's or disable testing for the yet unmodified rules. And land it to 3.0 otherwise diff would became pretty big so it would be impossible to review properly.

@markelog OK, I will make it green soon.

Babel moved comment attachment to the parser instead of estraverse to allow for comments in empty blocks - check out the recent babel-eslint changes


fade_brightness() method throws many different LookupError exceptions

The fade_brightness() method works > 80% of the time but has many different exception messages when filter_monitors throws a LookupError. They all seem to be caused by unexpected output from ddcutil. Here's three examples and my guesses as to what could be happening. The monitor cable is attached all throughout these exceptions and I have an image on the screen.
The first few API calls worked, then one that seems to have worked but the (possibly) cached value does not match the brightness value specified. Then it looks like the display is gone.
>>> sbc.set_brightness(60, display=1)
>>> sbc.fade_brightness(25, interval=1, increment=5, display=1)
[25]
>>> sbc.fade_brightness(60, interval=1, increment=5, display=1)
[60]
>>> sbc.fade_brightness(25, interval=0.75, increment=5, display=1)
[24]
>>> sbc.fade_brightness(60, interval=0.75, increment=5, display=1)
[24] # This did not fade to 60%. Both the output value and the percieved brightness did not change.
>>> sbc.fade_brightness(60, interval=0.75, increment=5, display=1)
[60]
>>> sbc.fade_brightness(60, interval=0.25, increment=5, display=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 251, in fade_brightness
    return get_brightness(**kwargs)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 44, in get_brightness
    return __brightness(display=display, method=method, meta_method='get', verbose_error=verbose_error)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 769, in __brightness
    raise ScreenBrightnessError(msg)
screen_brightness_control.helpers.ScreenBrightnessError:
        filter_monitors -> LookupError: no monitors found with name/serial/model/edid/index of 1

The next exception seems like the I2C device just...disappears.
>>> sbc.fade_brightness(65, display=1)
No monitor detected on I2C bus /dev/i2c-8
Exception in thread Thread-18:
Traceback (most recent call last):
  File "/home/lee/.pyenv/versions/3.9.7/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/home/lee/.pyenv/versions/3.9.7/lib/python3.9/threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 180, in fade
    monitor.set_brightness(val, no_return=True)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 481, in set_brightness
    self.method.set_brightness(value, display=self.index)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/linux.py", line 1150, in set_brightness
    subprocess.check_call(
  File "/home/lee/.pyenv/versions/3.9.7/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ddcutil', 'setvcp', '10', '21626', '-b', '8', '--sleep-multiplier=0.5']' returned non-zero exit status 1.
[32]

And finally...it looks like the I2C device is back but it seems that the EDID data is passed in hex to the index lookup rather then an int. If it helps this data is different if I export the EDID through the nvidia-settings application. The cached brightness value looks like it updated for a few loops and stopped with the exception.
>>> sbc.fade_brightness(65, display=1)
Exception in thread Thread-20:
Traceback (most recent call last):
  File "/home/lee/.pyenv/versions/3.9.7/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/home/lee/.pyenv/versions/3.9.7/lib/python3.9/threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 180, in fade
    monitor.set_brightness(val, no_return=True)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 479, in set_brightness
    self.get_info()
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 575, in get_info
    info = filter_monitors(display=identifier[1], method=self.method.__name__)[0]
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 691, in filter_monitors
    raise LookupError(msg)
LookupError: no monitors found with name/serial/model/edid/index of '00ffffffffffff0006102792df0e0d170d170104b53c2278226fb1a7554c9e250c5054000000010101010
10101010101010101010101565e00a0a0a029503020350055502100001a1a1d008051d01c204080350055502100001c000000ff004330324b4733395a463247430a000000fc005468756e64657
2626f6c740a20014e' with method of "DDCUtil"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 251, in fade_brightness
    return get_brightness(**kwargs)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 44, in get_brightness
    return __brightness(display=display, method=method, meta_method='get', verbose_error=verbose_error)
  File "/home/lee/.local/lib/python3.9/site-packages/screen_brightness_control/__init__.py", line 769, in __brightness
    raise ScreenBrightnessError(msg)
screen_brightness_control.helpers.ScreenBrightnessError:
        filter_monitors -> LookupError: no monitors found with name/serial/model/edid/index of 1

>>> sbc.get_brightness(display=1)
[44]

Well this is interesting.
The main thing here seems to be weird behaviour with DDCUtil and if we could see the stdout and stderr then I think it would explain alot.
I've pushed a commit to an issue18 branch that should print out stdout and stderr upon CalledProcessErrors so if you could try that and report back the output that would be great.
As for some of the other things you mentioned:

[...] it seems that the EDID data is passed in hex to the index lookup rather then an int.

This is intentional behaviour. fade_brightness works a little bit differently to set_brightness and get_brightness in that it uses the Monitor class to adjust each display's brightness individually. When Monitor.set_brightness is called it will try to refresh the information it has about itself to ensure it is up to date. Part of this is finding an data point (EDID, name, serial number or index) that is likely to be unique to the monitor and is a known value. This identifier is then passed to filter_monitors to return the current information about that monitor.
In this case, the EDID string was passed as the identifier to filter_monitors and a LookupError was raised because it could not find any monitors with that EDID string, meaning that DDCUtil.get_display_info likely did not report the displays correctly.
Secondly, in your first code example you show this snippet:
>>> sbc.fade_brightness(25, interval=0.75, increment=5, display=1)
[24]

Which demonstrates the brightness not quite being set to an accurate value. This is a bug I have been aware of for a little while and I think it has something to do with caches not being expired on time.

After doing a couple of unrelated tests on the library, I actually began to run into the above mentioned issues. Using the issue18 branch I was able to get the following outputs:
============STDOUT============
b''
============STDERR============
b'Verification failed for feature 10\n'

============STDOUT============
b'DDC communication failed for monitor on bus /dev/i2c-7\n'
============STDERR============
b''

I'm going to guess that these messages are the result of fade_brightness spamming the monitors and this has a chance of causing issues.
There are a few approaches to tackle this:

Use --noverify when setting brightness with DDCUtil (see ddcutil performance options)

But not always. Perhaps enable this option during fade_brightness but turn it off for a final check

If the command fails then retry after a short wait
If the command fails then increase the DDCUtil.sleep_multiplier value
A combination of the above

I'll muck about and see if these make a viable solution

Wooh, thanks. I took a week off open source things but I'm back. I'll rebuild and give it a shot.

lgtm! Hardware responded as expected. Thanks for this. I find a special kind of satisfaction getting obsolete Apple hardware to work better on non-apple computers.
>>> import screen_brightness_control as sbc
>>> print(sbc.fade_brightness(100, start=0, method='ddcutil', verbose_error=True))
[100]
>>> print(sbc.fade_brightness(0, method='ddcutil', verbose_error=True))
[0]
>>> print(sbc.fade_brightness(100, method='ddcutil', verbose_error=True))
[100]
>>> sbc.set_brightness(60, method='ddcutil', verbose_error=True)

Wonderful stuff. The issue18 branch was merged into main and pushed out in v0.14.0.
Thanks for raising the issue


Custom objective function

I'm trying to implement face detection with multi-task CNN based on this paper with keras as it's easy to create and use custom objective functions.
http://research.microsoft.com/en-us/um/people/chazhang/publications/wacv2014_ChaZhang.pdf

The objective function is computed as follows:
L = L1 (= loss of face/nonface decision) + L2 (= loss of head pose) + L3 (= loss of head landmarks)
I want L2 and L3 to be zero when the input is nonface.
As the loss for head pose / head landmarks depend on whether the input is face or not, it's not possible to just use Graph model with merge_mode 'sum'. So I merged the three outputs using add_output to obtain one output as a whole with a custom objective function. However, I've got a Theano error:
theano.gradient.DisconnectedInputError: grad method was asked to compute the gradient with respect to a variable that is not part of the computational graph of the cost, or is used only by a non-differentiable operator: <TensorType(float64, 4D)>.

I suspect the problem is Theano cannot compute the gradient for loss function as it involves subtensor.
Are there any ways to work around it?
Here's my model and custom objective function.
    # Model construction
    graph = Graph()
    graph.add_input(name='input', ndim=4)

    graph.add_node(Convolution2D(32, 1, 5, 5), name='conv1', input='input')
    graph.add_node(Activation('relu'), name='activation1', input='conv1')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool1', input='activation1')

    graph.add_node(Convolution2D(32, 32, 3, 3), name='conv2', input='pool1')
    graph.add_node(Activation('relu'), name='activation2', input='conv2')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool2', input='activation2')

    graph.add_node(Convolution2D(24, 32, 3, 3), name='conv3', input='pool2')
    graph.add_node(Activation('relu'), name='activation3', input='conv3')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool3', input='activation3')

    graph.add_node(Flatten(), name='flattened', input='pool3')
    graph.add_node(Dense(64 * 24, 512), name='dense')

    # Face/nonface
    # 0: nonface / 1: face
    graph.add_node(Dense(512, 128), name='dense11', input='dense')
    graph.add_node(Dropout(0.5), name='drop11', input='dense11')
    graph.add_node(Dense(128, 2), name='dense12', input='drop11')

    # Face noise
    graph.add_node(Dense(512, 128), name='dense21', input='dense')
    graph.add_node(Dropout(0.5), name='drop21', input='dense12')
    graph.add_node(Dense(128, 2), name='dense22', input='drop21')

    # Face landmarks
    graph.add_node(Dense(512, 256), name='dense31', input='dense')
    graph.add_node(Dropout(0.5), name='drop31', input='dense31')
    graph.add_node(Dense(256, 100), name='dense32', input='drop31')
    graph.add_node(Dropout(0.5), name='drop32', input='dense32')
    graph.add_node(Dense(100, 5), name='dense33', input='drop32')

    graph.add_output(name='output', inputs=['dense12', 'dense22', 'dense33'])
    graph.compile('sgd', {'output': loss})
    graph.fit({'input': X, 'output': y})

    def loss(y_true, y_pred):
        is_face_true, is_face_pred = y_true[:2], y_pred[:2]
        face_pose_true, face_pose_pred = y_true[2:7], y_pred[2:7]
        face_landmarks_true, face_landmarks_pred = y_true[7:17], y_pred[7:17]
        _loss = binary_crossentropy(is_face_true, is_face_pred)

        # additional loss for pose and landmarks if face
        return T.switch(T.lt(y_pred[0], y_pred[1]), _loss,
              _loss + categorical_crossentropy(face_pose_true, face_pose_pred)+mean_squared_error(face_landmarks_true, face_landmarks_pred))

for readability, I suggest that you wrap the sequential branches of your model in a layers.containers.Sequential. This will help you debug the graph topology.
it appears that your graph is effectively disconnected, i.e. the output cannot be traced back to the input.

I believe your error is here:
    graph.add_node(Dense(512, 128), name='dense21', input='dense')
    graph.add_node(Dropout(0.5), name='drop21', input='dense12')

dense21 is created but never used. I believe it should be used as the input of drop21 (instead of dense12).

Thanks for the fast reply!
I fixed what you pointed out but I still get the same error (theano.gradient.DisconnectedInputError).
It seems that Theano can't compute the gradient for this part.
is_face_true, is_face_pred = y_true[:2], y_pred[:2]
_loss = binary_crossentropy(is_face_true, is_face_pred)

I think one way to get around this problem is using weighted sum for loss function that depends on the prediction output. How could you achieve that?

Can you use a basic Sequential model with this loss function to make sure that the problem is with the loss function and not your Graph topology? Your error is typical of Graph topology issues.

I've rewritten the model using layers.containers.Sequential (and some fixes).
However, I still get the same error.
    # Model construction
    graph = Graph()
    graph.add_input(name='input', ndim=4)

    graph.add_node(Convolution2D(32, 1, 5, 5), name='conv1', input='input')
    graph.add_node(Activation('relu'), name='activation1', input='conv1')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool1', input='activation1')

    graph.add_node(Convolution2D(32, 32, 3, 3), name='conv2', input='pool1')
    graph.add_node(Activation('relu'), name='activation2', input='conv2')

    graph.add_node(Convolution2D(24, 32, 3, 3), name='conv3', input='activation2')
    graph.add_node(Activation('relu'), name='activation3', input='conv3')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool3', input='activation3')

    graph.add_node(Flatten(), name='flattened', input='pool3')
    graph.add_node(Dense(64 * 24, 512), name='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 128))
    seq.add(Dropout(0.5))
    seq.add(Dense(128, 2))
    graph.add_node(seq, name='output1', input='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 128))
    seq.add(Dropout(0.5))
    seq.add(Dense(128, 5))
    graph.add_node(seq, name='output2', input='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 256))
    seq.add(Dropout(0.5))
    seq.add(Dense(256, 100))
    seq.add(Dropout(0.5))
    seq.add(Dense(100, 10))
    graph.add_node(seq, name='output3', input='dense')

    graph.add_output(name='output', inputs=['output1', 'output2', 'output3'])

Can you use your custom loss function on a simple model such as:
seq = models.Sequential()
seq.add(Dense(512, 10))

If that yields the same error, then the problem is the loss function. Hard to tell otherwise.

Looks like you may be missing an input here:
graph.add_node(Flatten(), name='flattened', input='pool3')
graph.add_node(Dense(64 * 24, 512), name='dense')

With input:
graph.add_node(Flatten(), name='flattened', input='pool3')
graph.add_node(Dense(64 * 24, 512), name='dense', input='flattened')

Thank you @awentzonline for pointing out my mistake!
I've fixed it with some additional fixes. I've ended up a model like this:
    graph = Graph()
    graph.add_input(name='input', ndim=4)

    graph.add_node(Convolution2D(32, 1, 5, 5, border_mode='same'), name='conv1', input='input')
    graph.add_node(Activation('relu'), name='activation1', input='conv1')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool1', input='activation1')

    graph.add_node(Convolution2D(32, 32, 3, 3, border_mode='same'), name='conv2', input='pool1')
    graph.add_node(Activation('relu'), name='activation2', input='conv2')

    graph.add_node(Convolution2D(24, 32, 3, 3, border_mode='same'), name='conv3', input='activation2')
    graph.add_node(Activation('relu'), name='activation3', input='conv3')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool3', input='activation3')

    graph.add_node(Flatten(), name='flattened', input='pool3')
    graph.add_node(Dense(64 * 24, 512), name='dense', input='flattened')

    seq = containers.Sequential()
    seq.add(Dense(512, 2))
    graph.add_node(seq, name='output1', input='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 5))
    graph.add_node(seq, name='output2', input='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 10))
    graph.add_node(seq, name='output3', input='dense')

    graph.add_output(name='output', inputs=['output1', 'output2', 'output3'])
    graph.compile('sgd', {'output': loss})
    graph.fit({'input': X, 'output': y}, batch_size=32, nb_epoch=100)

Now I get an input dimension error:
ValueError: Input dimension mis-match. (input[0].shape[0] = 17, input[1].shape[0] = 5)
Apply node that caused the error: Elemwise{mul,no_inplace}(Elemwise{true_div,no_inplace}.0, Subtensor{int64:int64:}.0)
Inputs types: [TensorType(float64, col), TensorType(float64, matrix)]
Inputs shapes: [(17, 1), (5, 17)]
Inputs strides: [(8, 8), (136, 8)]
Inputs values: ['not shown', 'not shown']

Happy to help when I can. I'd say your last issue is probably due to whatever shape y has.

Thanks to your help, I've found that the following model and objective function works.
def loss(y_true, y_pred):
    is_face_true, is_face_pred = y_true[:2], y_pred[:2]
    face_pose_true, face_pose_pred = y_true[2:7], y_pred[2:7]
    face_landmarks_true, face_landmarks_pred = y_true[7:17], y_pred[7:17]
    l1 = binary_crossentropy(is_face_true, is_face_pred)
    l2 = categorical_crossentropy(face_pose_true, face_pose_pred)
    l3 = mean_squared_error(face_landmarks_true, face_landmarks_pred)
    l = T.concatenate([l1, l2, l3])
    return l

    # model
    graph = Graph()
    graph.add_input(name='input', ndim=4)

    graph.add_node(Convolution2D(32, 1, 5, 5, border_mode='same'), name='conv1', input='input')
    graph.add_node(Activation('relu'), name='activation1', input='conv1')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool1', input='activation1')

    graph.add_node(Convolution2D(32, 32, 3, 3, border_mode='same'), name='conv2', input='pool1')
    graph.add_node(Activation('relu'), name='activation2', input='conv2')

    graph.add_node(Convolution2D(24, 32, 3, 3, border_mode='same'), name='conv3', input='activation2')
    graph.add_node(Activation('relu'), name='activation3', input='conv3')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool3', input='activation3')

    graph.add_node(Flatten(), name='flattened', input='pool3')
    graph.add_node(Dense(64 * 24, 512), name='dense', input='flattened')

    seq = containers.Sequential()
    seq.add(Dense(512, 17))
    graph.add_node(seq, name='o', input='dense')

    graph.add_output(name='output', input='o')
    graph.compile('sgd', {'output': loss})
    graph.fit({'input': X, 'output': y}, batch_size=32, nb_epoch=10)

However, when I "split" the graph in seq part, I get ValueError:
    seq = containers.Sequential()
    seq.add(Dense(512, 2))
    graph.add_node(seq, name='output1', input='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 5))
    graph.add_node(seq, name='output2', input='dense')

    seq = containers.Sequential()
    seq.add(Dense(512, 10))
    graph.add_node(seq, name='output3', input='dense')

    graph.add_output(name='output', inputs=['output1', 'output2', 'output3'], merge_mode='concat')

# error

ValueError: operands could not be broadcast together with shapes (2,2) (2,17) (2,2)
Apply node that caused the error: IncSubtensor{InplaceInc;:int64:}(Alloc.0, Elemwise{Composite{(AND(GE(i0, i1), LE(i0, i2)) * (((i3 * i4 * i5) / i6) + ((i7 * i8) / (i9 * i10))))}}.0, Constant{2})
Inputs types: [TensorType(float64, matrix), TensorType(float64, matrix), Scalar(int64)]
Inputs shapes: [(32, 2), (2, 17), ()]
Inputs strides: [(16, 8), (136, 8), ()]
Inputs values: ['not shown', 'not shown', 2]

Aha, you are also interested in implementing this paper. Here is another good multi-task landmark detection paper CUHK-TCDCN which comes with a larger dataset but uses locally connected layer instead of the convolutional layer with shared weights.

@rex-yue-wu Let me know if you successfully implement it! Also, thanks for the link I'll read it later!

So, what is:

your final complete model
the shape of your inputs

This is my final (but simplified) model.
Input is 20000 32 x 32 grayscale images (20000 x 1 x 32 x 32)
output is 20000 x 17.
    # model
    graph = Graph()
    graph.add_input(name='input', ndim=4)

    graph.add_node(Convolution2D(32, 1, 5, 5, border_mode='same'), name='conv1', input='input')
    graph.add_node(Activation('relu'), name='activation1', input='conv1')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool1', input='activation1')

    graph.add_node(Convolution2D(32, 32, 3, 3, border_mode='same'), name='conv2', input='pool1')
    graph.add_node(Activation('relu'), name='activation2', input='conv2')

    graph.add_node(Convolution2D(24, 32, 3, 3, border_mode='same'), name='conv3', input='activation2')
    graph.add_node(Activation('relu'), name='activation3', input='conv3')
    graph.add_node(MaxPooling2D(poolsize=(2, 2)), name='pool3', input='activation3')

    graph.add_node(Flatten(), name='flattened', input='pool3')
    graph.add_node(Dense(64 * 24, 512), name='dense', input='flattened')

    seq = containers.Sequential()
    seq.add(Dense(512, 17))
    graph.add_node(seq, name='o', input='dense')

    graph.add_output(name='output', input='o')


Sign Up Permitted Parameters gets Passed but Never saves succesfully to Database

I am adding Nickname, Name and a Role field to the new user registration Parameter. This new added Parameters I passed show up in my Rails console. As seen below the SQL Insert State shows the following.
SQL (23.3ms)  INSERT INTO "users" ("name", "nickname", "email", "encrypted_password", "role_id", "tokens", "uid", "created_at", "updated_at", "confirmation_token", "confirmation_sent_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) RETURNING "id"  [["name", "Neil Hanekom"], ["nickname", "Neil"], ["email"<EMAIL_ADDRESS>["encrypted_password", "$2a$10$z/Jpz8shRfXa9E46O70uk.F6NLNVXcqmSKLkFs6i/e1w39soYOwKS"], ["role_id", 1], ["tokens", "{}"], ["uid"<EMAIL_ADDRESS>["created_at", "2016-07-28 15:31:45.666805"], ["updated_at", "2016-07-28 15:31:45.666805"], ["confirmation_token", "JJzkE_2B6yWPrNhg43HC"], ["confirmation_sent_at", "2016-07-28 15:31:45.667027"]]
However the new Parameters never gets saved. Why would this happen??

Having the same issue here

This worked for me:
In base_controller.rb:
# ...
include DeviseTokenAuth::Concerns::SetUserByToken
prepend_before_filter :configure_permitted_parameters, if: :devise_controller?

protected

def configure_permitted_parameters
  devise_parameter_sanitizer.for(:sign_up) << :name
  ## duplicate the line
  # devise_parameter_sanitizer.for(:sign_up) << :nickname
  # devise_parameter_sanitizer.for(:sign_up) << :role
end
# ...

Workaround posted, closing for now.


Optimize multiline diff for (some metric of) edit-distance

I'm trying to use pretty-assertions for snapshot testing. I compare the snapshotted multiline-string with the new value using the PrettyString trick from #24.
When the actual result got indented (compared to the snapshot), the diff output became very confused:

The reason is that two unrelated lines (Foo's and Bar's lorems) got matched up in the first (line-based) comparison pass, which should instead realize there's a new line Some(, followed by a few modified lines, followed by a new line ).
#[test]
fn test_multiline_diff() {
    assert_eq!(PrettyString(r#"Foo {
    lorem: "Hello World!",
    ipsum: 42,
    Bar {
        lorem: "Hello World!",
    }
}"#), PrettyString(r#"Some(
    Foo {
        lorem: "Hello World!",
        ipsum: 42,
        Bar {
            lorem: "Hello World!",
        }
    }
)"#));
}

I agree, that the presentation is confusing - despite being a correct solution, it doesn´t seem to be an optimal solution.
You already mentioned, that the current heuristic algorithm is doing two passes and the first is line-based, while the second pass is character-based. Thats the problem: The first pass doesnt look at whitespaces (or any metric of edit-distance)...
I´m open for suggestions about a better algorithm. ^^
(This must have been solved before.. ;-))

Thats the problem: The first pass doesnt look at whitespaces

Another way of saying this, I think, is that the difference crate is not line-aware, and does a LCS diff as if the entire input is one line. This is quite right for, e.g. a -/+ pair of lines, to show the specific changes. But it doesn't present good output for multiline inputs the way Myers or Patience will. (Consensus seems to be that Patience is generally the best we can do for code and similar things, but all diff systems have pathological cases.)
Maybe it's worth digging into diff-so-fancy, which seems to be the 'solved before' we're looking for. It looks like the general approach is to start with Patience, and then supplement its output with intra-line diffs based on a heuristic like '-/+ pairs get intra-line LCS diffs'. I suspect these heuristics are the difficult part.

I deleted a previous comment that was wildly inaccurate.
What we actually want here is

A subjectively better diff algorithm than the difference crate's plain LCS, like Patience or Histogram, which is a faster version of Patience: look through the figures in https://rdcu.be/bWfbo for a sense of how much better it is than Myers, or tell Git to use diff.algorithm = histogram and try it out.
Whitespace-insensitivity in the first/line-based pass. A newtype wrapper over &str that implements (Partial)Ord by ignoring consecutive whitespace would do this without infecting the diff algorithm code. Is this desirable for all pretty-assertions, or would it need to be configurable? I would guess it could be applied everywhere. Any absolute differences in the entire debug string, including whitespace, should still cause test failure IMO. A 'no diff output, which indicates changes in whitespace only' message would help.


:seedling: add support for 1.10 in e2e

Add support for CAPM3 1.10 development in CI.

/hold
For 1.9 to be cut, and also clarity on the content.

For 1.9 to be cut, and also clarity on the content.

1.9 should be fine as-is (without this PR) contract wise.
Discussed with @kashifest , this seems to be fine regarding 1.10 metadata/contracts etc. Need to sync with @adilGhaffarDev tomorrow, as release-1.8 cut was messy around these.
Holding until final clear and 1.9 is cut.
/cc @kashifest @adilGhaffarDev

/lgtm

/test metal3-centos-e2e-integration-test-main

This is expected to fail until 1.9 is out, and even then we need to merge dev-env pr first. https://github.com/metal3-io/metal3-dev-env/pull/1485

/approve

/test metal3-centos-e2e-integration-test-main metal3-ubuntu-e2e-integration-test-main

/test metal3-centos-e2e-integration-test-main metal3-ubuntu-e2e-integration-test-main

/lgtm
/approve


Test is failing without any error message

I have this too

Is it failing at that step? If so it's probably that the app credentials are missing the product read permission.
If not, then I'd need to know exactly where it fails.

I have the same issue, I just went into my Shopify and gave the app credentials:
Product Listing and Products both read access just to be on the safe side and still fails and gets the same error message

Hiya, I think I've found what the problem was. It looks like the curl request was failing silently. This could happen for a multitude of reason but the one that comes to mind would be an invalid store configuration variable.
Before, we assumed that the protocol (https://) was not part of this setting. If it was in there, the curl request would fail silently.
I'll also add code to make this not error out when this happens in the future, one shouldn't need to care about this.


Added config to run tests on travis-ci.com

Also you can register and add project on https://codecov.io

I think that it is better to have tests configuration only in one place, currently it is in tox.ini, and it is possible to reuse it by using tool like tox-travis, example: https://github.com/vmagamedov/kinko/blob/master/.travis.yml
Is it worth to cache installed packages? This cache will be downloaded each time by Travis from S3 and Travis doesn't recommend to cache such things, but only CPU heavy steps, like sources compilation.


Crashes after zooming out when moved using capture cursor.

(Branch: opengl)
When moved, using capture cursor, the window crashes when zooming out.
thread 'main' panicked at 'index out of bounds: the len is 50 but the index is<PHONE_NUMBER>3709551610'

This was fixed when zoom in on centre was removed.
If it will be added again, this issue should be resolved first.


Threading changes in 3.2.4-rc3 break C# threading

Godot version:
3.2.4-rc3
OS/device including version:
Ubuntu
Issue description:
This PR https://github.com/godotengine/godot/pull/45618 breaks C# threading.
Steps to reproduce:
Honestly it's hard to tell, I have a number of Task running in the background
in order to handle requests. It worked in 3.2.1, then other issues prevented me from
using newer version of Godot until 3.2.4-rc2 where everything just worked.
Minimal reproduction project:
Will try to make one that causes the issue and then post it.
Just wanted this PR to get flagged as soon as possible, would hate to see this end up in 3.2.4 final.

This PR #45618 breaks C# threading.

Did you confirm that by testing a local revert, or is it a hunch based on the similarity of the PR and the issue you're experiencing?

What I did so far is open the project in rc2 and see it work,
open the project in rc3 and see it fail and then repeat.
As for building it from source, given my geographical location it will take some time for
me to download everything first.

I can do a bisect myself if you can provide a project that reproduces the issue (which would likely be needed in any case to debug and fix the issue).

Same situation exists in my project. I use interactive loader inside the thread. All textures turn black box when loading is complete.
It works fine in the 3.2.4 rc2.
I've been having some time problem at work for a while. I could not produce a sample project in a simple structure. If no one has done it, I'll create a sample project over the weekend.

I've done some tests with Godot Thread and ResourceLoader.LoadInteractive in the thread but couldn't reproduce any kind of breakage with 3.2.4 RC 3. So we'll really need a reproduction project indeed to debug further. I'm probably going to build 3.2.4 RC 4 tomorrow so there's a risk we won't have a fix for this issue by then... and then there might be a need for another RC 5 afterwards.

Sorry. When I tried it again, I realized that my problem was not thread sourced but weakref. I will create a new issue topic title related to the problem.

Pinned it down to calling GetStringSize in another thread.
The attached project works great in rc2, crashes in rc3
Issue46550.zip
Relevant code:
using Godot;
using System;
using System.Threading.Tasks;
public class Node2D : Godot.Node2D
{
    public override void _Ready()
    {
        var task = Task.Factory.StartNew(() => ProcessStuff(), TaskCreationOptions.LongRunning);
    }

    public void ProcessStuff()
    {

        var fontData = (DynamicFontData)GD.Load("res://Ubuntu-B.ttf");
        var font = new DynamicFont()
        {
            FontData = fontData,
            Size = 16,
        };

        font.GetStringSize("Let it crash");
    }
}

Hope this helps.

Pinned it down to calling GetStringSize in another thread.
The attached project works great in rc2, crashes in rc3
Issue46550.zip
Relevant code:
using Godot;
using System;
using System.Threading.Tasks;
public class Node2D : Godot.Node2D
{
    public override void _Ready()
    {
        var task = Task.Factory.StartNew(() => ProcessStuff(), TaskCreationOptions.LongRunning);
    }

    public void ProcessStuff()
    {

        var fontData = (DynamicFontData)GD.Load("res://Ubuntu-B.ttf");
        var font = new DynamicFont()
        {
            FontData = fontData,
            Size = 16,
        };

        font.GetStringSize("Let it crash");
    }
}

Hope this helps.

Thanks, that's a great reproduction project. I couldn't trigger the issue on Linux so it might be macOS-specific.

I do not use MacOS
I use Ubuntu 20.10...
Am able to reproduce it with the sample 100% of the times.

Welp my bad, I got mixed up with the other macOS issues.

Do you get any backtrace when it crashes?

Indeed, that's helpful! I'll check it.

Will do that when I get back to the office.

I set up a Ubuntu 20.04 VM to test (note: not 20.10, but I thought it might be close enough), and install Mono <IP_ADDRESS> and MSBuild 16.6 from the current upstream repo: https://www.mono-project.com/download/stable/
Sadly still can't reproduce the issue :|

I added the ._sc_ file and confirmed it's running in self contained mode.
Unzipped the project in a new folder and opened it
./Godot_v3.2.4-rc3_mono_x11.64 -v --editor /home/timothy/TMP/Issue46550/project.godot

The output and mono log are attached.
Mono
Mono JIT compiler version <IP_ADDRESS> (tarball Mon Feb 22 17:33:28 UTC 2021)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug
	Interpreter:   yes
	LLVM:          yes(610)
	Suspend:       hybrid
	GC:            sgen (concurrent by default)

msbuild:
Microsoft (R) Build Engine version 16.6.0 for Mono
Copyright (C) Microsoft Corporation. All rights reserved.

<IP_ADDRESS>0

monolog+output.zip

Thanks! The crash log lacks debug symbols (expected as official binaries are built without them, you'd have to build from source to get them), but whatever it includes is already useful:
Thread 9 (Thread 0x7f99849ff640 (LWP 179050)):
#0  0x00007f99a775068f in __GI___wait4 (pid=pid@entry=179059, stat_loc=stat_loc@entry=0x7f99849fd294, options=options@entry=0, usage=usage@entry=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
#1  0x00007f99a775060b in __GI___waitpid (pid=pid@entry=179059, stat_loc=stat_loc@entry=0x7f99849fd294, options=options@entry=0) at waitpid.c:38
#2  0x0000000000f12ed2 in dump_native_stacktrace (signal=signal@entry=0x3cd60db "SIGSEGV", mctx=mctx@entry=0x7f9974002370) at mini-posix.c:1112
#3  0x0000000000f12fe9 in mono_dump_native_crash_info (signal=signal@entry=0x3cd60db "SIGSEGV", mctx=mctx@entry=0x7f9974002370, info=info@entry=0x0) at mini-posix.c:1156
#4  0x0000000000f05caf in mono_handle_native_crash (signal=0x3cd60db "SIGSEGV", mctx=mctx@entry=0x7f9974002370, info=info@entry=0x0) at mini-exceptions.c:3423
#5  0x0000000000e7d459 in altstack_handle_and_restore (ctx=0x7f9974002370, obj=0x0, flags=<optimized out>) at exceptions-amd64.c:879
#6  0x00007f99a71b7989 in ?? () from /lib/x86_64-linux-gnu/libGLdispatch.so.0
#7  0x00007f99849fe230 in ?? ()
#8  0x0000000002e4beb8 in RasterizerStorageGLES2::texture_create() ()
#9  0x0000000000000002 in ?? ()
#10 0x00000000075ee0d0 in ?? ()
#11 0x00007f997400e5e0 in ?? ()
#12 0x00007f99849fe230 in ?? ()
#13 0x00007f9974015460 in ?? ()
#14 0x0000000000000008 in ?? ()
#15 0x0000000000000002 in ?? ()
#16 0x00000000018e2ea2 in VisualServerWrapMT::texture_create() ()
#17 0x00007f997400e5e0 in ?? ()
#18 0x00007f99849fe230 in ?? ()
#19 0x00007f9974015460 in ?? ()
#20 0x0000000000000008 in ?? ()
#21 0x0000000000000002 in ?? ()
#22 0x00007f997400e5e0 in ?? ()
#23 0x00007f997400e5e0 in ?? ()
#24 0x00007f99849fe230 in ?? ()
#25 0x00007f9974015460 in ?? ()
#26 0x0000000000000008 in ?? ()
#27 0x0000000000000002 in ?? ()
#28 0x0000000001c210ab in ImageTexture::ImageTexture() ()
#29 0x0000000000000001 in ?? ()
#30 0x00007f9974015480 in ?? ()
#31 0x00007f99849fe230 in ?? ()
#32 0x000000000348d4a0 in Ref<ImageTexture>::instance() ()
#33 0x00007f99849fe230 in ?? ()
#34 0x0000000001bd1439 in Ref<Image>::ref_pointer(Image*) [clone .isra.0] [clone .lto_priv.4] [clone .lto_priv.0] ()
#35 0x00007f9974022dc0 in ?? ()
#36 0x00007f9974015220 in ?? ()
#37 0x00007f99849fe230 in ?? ()
#38 0x0000000001da6a7f in DynamicFontAtSize::_bitmap_to_character(FT_Bitmap_, int, int, float) ()
#39 0x00007f997401ff28 in ?? ()
#40 0x00007f990000000d in ?? ()
#41 0x0000000000000001 in ?? ()
#42 0x0000000000000000 in ?? ()

It seems to confirm a crash triggered in RasterizerStorageGLES2::texture_create() due to DynamicFont.
That sounds quite similar to #46679, and instead of platform-specific, this might be a GPU driver specific issue.

godot-3.2.4-rc-log.zip
From the debug version you provided

Left few stack traces from macOS in the #46679 (same issue), seems like it's always crashing on OpenGL texture API function called from the thread.

@akien-mga you cannot reproduce it locally, maybe we have different GPU vendors or drivers?
Mine:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.39       Driver Version: 460.39       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 1060    Off  | 00000000:01:00.0  On |                  N/A |
| 38%   42C    P8    12W / 120W |   1476MiB /  3016MiB |      9%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+


Pass unknown args to inner command

Fixes: https://github.com/con/duct/issues/51
It works-- but I'm marking as a draft to investigate what happens if the inner command just happens to have the same named arg as duct does.

I think this would break for inner commands that share arg names with duct. closing.


Type d is generated even without cellDates: true for ODS files

https://docs.sheetjs.com/#data-types states

Type d is the Date type, generated only when the option cellDates is passed.

This piece of code suggests that the Date time is generated even without that option, if the output file is ods:
const XLSX = require('xlsx');

const wb = XLSX.utils.book_new();

const ws = XLSX.utils.json_to_sheet([]);

XLSX.utils.book_append_sheet(wb, ws, 'Bug sheet');

ws.A1 = { t: 'd', v: new Date('2018-11-30T23:30:00Z') };

XLSX.writeFile(wb, 'date.ods');

The output file contains
<table:table-cell office:value-type="date" office:date-value="2018-11-30T23:30:00.000Z" table:style-name="ce1">
  <text:p>2018-11-30T23:30:00.000Z</text:p>
</table:table-cell>

Saw at https://docs.sheetjs.com/#writing-options

cellDates only applies to XLSX output

Perhaps worth repeating that in the Data Types section?

Situation is described below.  If you think the README is missing something, please send a PR with the recommended changes:
ODS
ODS has a native date type, with office:value-type="date".
When reading a date cell, cellDates is used to determine whether a proper Date object or a datecode number is generated.
When writing a date, SheetJS will ignore the cellDates property and generate a proper ODS date.
XLS / XLSB
XLS / XLSB do not have a native date type.  It is always numbers with a format that uses date tokens.
When reading a cell, the number format is checked.  If the format looks like a date, cellDates determines whether a Date or Number cell is generated.
When writing a date, SheetJS will ignore the cellDates property and write a Number cell.  If a date format is not specified, the default format 14 will be used.
XLSX
XLSX supports the XLS-style format codes (t="n")as well as a date type (t="d").  Note that the date type ignores the timezone metadata, so it has the same problems as date codes)
When reading a date cell, cellDates determines whether a Date or Number cell is generated.
When reading a number cell, the number format is checked.  If the format looks like a date, cellDates determines whether a Date or Number cell is generated.
When writing a date, SheetJS will check the cellDates property to determine whether to write a Date cell or a Number cell


Do we have to copy the source code file in order to customize BrowseFragment?

Hello,
Thank you for your amazing article.
May I ask the question that in the title of this issue?
Since it would be difficult if we update the leanback lib to the latest version.
I met a problem that I want to control badge visibility base on the header drawer, however, I don't know how to do it since the headerFragment don't give out a way to verify is the drawer open or close.
Thank you in advance for your kind answer!

Have you tried using Sofa instead? This is just the source code that was written for the articles, while Sofa is a full fledged Android TV library.

No, it is not a personal project..I could not use it....may I ask how to do it without Sofa?

OK. Back to your question, it depends on what you're trying to do. If it's mostly showing/hiding stuff, then using Reflection is sufficient for most of the cases. Otherwise yes, you need to copy the source code and modify it to your needs.


Fix num_graphemes when used with a prefix

When used with num_bytes smaller than the length of the string and when the Token does not have a char_map (created outside of the tokenizer perhaps), the output of num_graphemes_from_bytes was wrong - it would return the length of the internal string instead of the number of bytes that was asked.
Pull Request
What does this PR do?
Fixes #63

PR checklist
Please check if your PR fulfills the following requirements:

[x] Does this PR fix an existing issue?
[x] Have you read the contributing guidelines?
[x] Have you made sure that the title is accurate and descriptive of the changes?

I think that your issue is not about returning bytes count instead of char count but about using char instead of graphemes. In the deunicode normalizer we may iter over chars token.word.chars() instead of  graphemes token.word.graphemes(true). At the end we are interested in how many character we have to highlight. 🤔

Hey @Samyak2! I think that your issue is not about returning bytes count instead of char count but about using char instead of graphemes. In the deunicode normalizer we may iter over chars token.word.chars() instead of graphemes token.word.graphemes(true). In the end we are interested in how many character we have to highlight. thinking

Hmm you're right. If we only need chars (and not graphemes), it makes this a lot simpler. Instead of having an extra char_map, we can just use char_indices and chars() here.
Thank you! I'll push a commit in this PR fixing these issues.

I have made some changes in the latest commit. Instead of graphemes, I have used .chars() instead. The char_map stores the number of bytes in normalized string for each char in original string.
The bug was in the case where char_map wasn't initialized. I have modified it to return the number of chars for given bytes in the current string.

Hey @Samyak2 !
I think it would be simplier to do a saturating_sub() on byte_index using each char_index. A take_while() on byte_index > 0 would do the job 🤔

Hey @Samyak2 ! I think it would be simplier to do a saturating_sub() on byte_index using each char_index. A take_while() on byte_index > 0 would do the job thinking

What do you mean by byte_index?
I initially tried a simple version like this:
                self.word
                    .char_indices()
                    .take_while(|(char_index, _)| {
                        *char_index < num_bytes
                    })
                    .count()

But it does not work for cases where a char is more than one byte, but we don't have enough bytes left to complete the char. For example, token.num_chars_from_bytes(3) for the word Go💼od should return 2 since we only have enough bytes for 2 characters and the 3rd character will need 2+4 bytes. If we are sure that num_chars_from_bytes won't be called in such a case where we only have bytes to represent half (or any fraction) of a character, then we can use this (much) simpler solution.
@ManyTheFish what do you think?

Hey @Samyak2 ! I think it would be simplier to do a saturating_sub() on byte_index using each char_index. A take_while() on byte_index > 0 would do the job thinking

What do you mean by byte_index?
I initially tried a simple version like this:
                self.word
                    .char_indices()
                    .take_while(|(char_index, _)| {
                        *char_index < num_bytes
                    })
                    .count()

But it does not work for cases where a char is more than one byte, but we don't have enough bytes left to complete the char. For example, token.num_chars_from_bytes(3) for the word Go💼od should return 2 since we only have enough bytes for 2 characters and the 3rd character will need 2+4 bytes. If we are sure that num_chars_from_bytes won't be called in such a case where we only have bytes to represent half (or any fraction) of a character, then we can use this (much) simpler solution.
@ManyTheFish what do you think?

Hello @Samyak2,
I would say that the below code (your proposal) is acceptable because if I search gobriefc   highlighting Go💼 would be ok.
                self.word
                   .char_indices()
                   .take_while(|(char_index, _)| {
                       *char_index < num_bytes
                    })
                   .count()

But if you think that we should only highlight "overlapped" characters then I would propose.
                self.word
                   .chars()
                   .take_while(|char| {
                       let stop = char.len() > num_bytes;
                       num_bytes.saturating_sub(char.len());
                       stop
                    })
                   .count()

My proposal as an issue, if we have the word 💼 and I'm searching briefca nothing would be highlighted 🤔
what do you think?

                self.word
                   .chars()
                   .take_while(|c| {
                       let stop = c.len() > num_bytes;
                       num_bytes = num_bytes.saturating_sub(c.len());
                       stop
                    })
                   .count()

My proposal as an issue, if we have the word 💼 and I'm searching briefca nothing would be highlighted thinking
what do you think?

@ManyTheFish I had to make some changes to your proposal to make it pass all test cases:
                self.word
                   .chars()
                   .take_while(|&c| {
                       let stop = num_bytes >= c.len_utf8();
                       num_bytes = num_bytes.saturating_sub(c.len_utf8());
                       stop
                    })
                   .count()

But I hadn't thought of this when writing my proposal:

I would say that the below code (your proposal) is acceptable because if I search gobriefc highlighting Go💼 would be ok.

It makes a lot more sense to highlight it even if only a part of the char is available. I will implement this solution and commit. Thank you!


add stat_prefix field in sdsconfig grpc setting

https://github.com/istio/istio/issues/6411
integration shows stat_prefix field is required otherwise envoy fails to start

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: quanjielin
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: costinm
Assign the PR to them by writing /assign @costinm in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/cc @qiwzhang

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: quanjielin, wattli
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [wattli]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Fix Grammar Mistake

Hi, you need to sign your commit.

I'm going to close it and re-open it to rekick the CI because it seems t be in a bit od a sad state now.


Lexing ( is incorrect

We need to take into account the lex state to delineate between ( being used for parentheses and ( being used for arguments. For example:
foo (bar), baz

currently will not parse correctly. Even worse it fails with a very confusing error message about multiple assignment.

Fixed on main by #522


Fix typo

Is it really a typo? In the original text, "simply" is an adverb that modifies an adjective. In your proposed change, "simply" has become an adverb that modifies the sentence.

"This file is a simply formatted set of fields describing the package's metadata." is grammatically questionable. If it is supposed to be and adjective, it should be simple. So either it is "just" a formatted set of fields or it is a simple formatted set of fields. Probably "a set of formatted fields" would be even more sensible. I don't think it's worth spending time on this though, feel free to just close this or do whatever you please :)

If we have

a simple formatted set of fields

then "simple" is an adjective modifying "set of fields". If, on the other hand, we have

a simply formatted set of fields

then "simply" is an adverb modifying "formatted". Since "formatted" here is an adjective, this means "a set of fields" is described by the adjective phrase "simply formatted". In other words, the formatting is simple.

feel free to just close this

I'm not a collaborator on this project, so I can't do that. Besides, it's up to the vcpkg team to decide what the intended meaning of the original sentence is and whether the current wording is clearly conveying that meaning.

Thanks for pointing out that this could be difficult to read for some! I felt like the most important word was actually "simple", not "formatted", so I revised it further to completely remove any confusion :)


feat: merge-insert supports inserting subset of columns

In https://github.com/lancedb/lance/pull/2639 we added support for updating subcolumns. In https://github.com/lancedb/lance/pull/3041 we added support for inserting subcolumns. This PR adds support for upserting them (or doing insert-if-not-exists).
Closes #2904
Example
import pyarrow as pa
import lance

table = pa.table({
    "id": range(3),
    "a": [1.0, 2.0, 3.0],
    "c": ["x", "x", "x"]
})
dataset = lance.write_dataset(table, "example")

# Upsert: when_matched_update_all + when_not_matched_insert_all
new_data = pa.table({
    "id": [2, 3],
    "c": ["y", "y"]
})
(
    dataset
    .merge_insert(on="id")
    .when_matched_update_all()
    .when_not_matched_insert_all()
    .execute(new_data)
)
dataset.to_table().to_pandas()

   id    a  c
0   0  1.0  x
1   1  2.0  x
2   2  3.0  y
3   3  NaN  y

# Insert-if-not-exists: when_not_matched_insert_all
new_data = pa.table({
    "id": [3, 4],
    "c": ["z", "z"]
})
(
    dataset
    .merge_insert(on="id")
    .when_not_matched_insert_all()
    .execute(new_data)
)
dataset.to_table().to_pandas()

   id    a  c
0   0  1.0  x
1   1  2.0  x
2   2  3.0  y
3   3  NaN  y
4   4  NaN  z

Codecov Report
Attention: Patch coverage is 83.83838% with 16 lines in your changes missing coverage. Please review.

Project coverage is 78.91%. Comparing base (f2906cf) to head (e7a79cf).
Report is 2 commits behind head on main.

Files with missing lines
Patch %
Lines

rust/lance/src/dataset/write/merge_insert.rs
83.83%
9 Missing and 7 partials :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3100      +/-   ##
==========================================
- Coverage   78.97%   78.91%   -0.06%
==========================================
  Files         246      246
  Lines       86313    86560     +247
  Branches    86313    86560     +247
==========================================
+ Hits        68162    68311     +149
- Misses      15328    15421      +93
- Partials     2823     2828       +5

Flag
Coverage Δ

unittests
78.91% <83.83%> (-0.06%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Example ?

Hi, the component looks interesting.
Any example to better show how widths should be calculated ?

@jbrodriguez Hi there. We will make sure to add an example as soon as we find the time to do so. Unfortunately, this might take a couple of weeks.
Is there any way to help you get a better understanding in the meantime? Maybe with an example of your own for which you need some insight?

Yeah its really unfortunate theres really no hope how to get it started . This seems like such a great component but I can't even get anything to show on the scroll.

@imjameshall We totally understand you. Ok, you got us motivated; you'll have an example by the end of the week :)

+1

Ok guys, as promised you can now play with the following example. Hope this will help you get started!


Upload gallery mostly blank

When I go to my website/u/ it's just a white page with the text "Uploading site" at the top and under it say's "Free space: 95.15 GB / 139.21 GB " (which is wrong) images upload fine but I just can't browse them.
I put the "u" folder and upload.php from inside src into the root of my site.
If I take my IP out of the config.php then nothing changes except it adds the text: "Your IP is blocked from access, whitelist this ip to gain access: (my ip)"

Do you get any console errors, wondering if something is failing front end wise?

Make sure you put in your domain correctly

If you're still getting this issue, could you send over your config? This may have something to do with invalid URLs configured, as Carson said.


Monorepo support

Clear and concise description of the problem
This won't work in a monorepo, let's say a pnpm/turborepo monorepo.
It only looks for a .gitignore in the cwd, but if use eslint-config-flat-gitignore in a workspace and i only use a .gitignore at the top-level of the monorepo, nothing will be found.
Suggested solution
Maybe use something like find-up to lookup for the .gitignore towards the top?
Alternative
No response
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guide.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Sure, PR welcome :)

Sure, PR welcome :)

This is great, but find-up is pure ESM start from 6.0.0, which breaks the cjs support of eslint-config-flat-gitignore, any suggestions?

@Superoryco
yes, i understand that, but it's not actually too bad.
If your project is not fully ESM yet, you can solve it just by setting eslint.config to ".mjs", so "eslint.config.mjs".
You can do so with eslint version >= 8.57.0 or using eslint-ts-patch.

@AndreaPontrandolfo Thank you for the tip, just noticed 😄


Workaround deltaTime bug

Calling love.timer.step() upon initialization properly sets up the delta time for the second call. This fixes #56

That was... quick. lol


netlink should not present WiFi interface as ARPHRD_IEEE80211

Windows build number: Microsoft Windows [Version 10.0.16299.248]
What you're doing and what's happening:
My laptop has a Atheros QCA9377 WiFi interface.
I am trying to run NFD 0.6.1 within Ubuntu environment. Part of this program uses netlink socket to enumerate network interfaces, and determine which network interfaces can be used for UDP multicast.
However, netlink reports that wifi0 interface has type ARPHRD_IEEE80211, which indicates the WiFi interface is in monitor mode, and causes NFD to conclude this network interface is unusable.
This is also observable with ip link command:
sunny@sunny5:~$ ip link show wifi0
17: wifi0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ieee802.11 3c:a0:67:f4:11:65

What should be happening instead:
The netlink socket should report wifi0 interface as ARPHRD_ETHER type.
As comparison, on a regular Ubuntu machine (Raspberry Pi 3 with Ubuntu Mate 16.04), WiFi interface is correctly presented as ARPHRD_ETHER:
sunny@pimate:~$ ip link show wlan0
4: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b8:27:eb:20:b5:e2 brd ff:ff:ff:ff:ff:ff

See also bug report with NFD: https://redmine.named-data.net/issues/4534

Pretty sure NFD is not going to fly because socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_ROUTE) here, which is 2249; nevermind ARPHRD_ETHER. As a separate matter WSL doesn't support arp either #2279, but I don't think that's the fundamental issue.
You can try editing the code and return InterfaceType::ETHERNET unilaterally and see what happens (I have not).
Someone else will have to rule on whether ARPHRD_IEEE80211 is "correct" here. I kind of suspect it is correct (for some definition of correct), but not enough to jump the gun.

Why is socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_ROUTE) a problem? Based on what I'm reading around, WSL does support NETLINK_ROUTE for link and address messages (RTM_NEWLINK and RTM_NEWADDR), which is all NFD needs. As a matter of fact, ethernet interfaces are detected correctly without any code modifications.
ARP has nothing to do with this, the "ARP" in the names of ARPHRD_* constants is historical I think.
I'm not sure what "correct" means in this context, but the Linux kernel reports wifi interfaces in station mode as ARPHRD_ETHER, I guess because you just send/receive regular ethernet frames on them. ARPHRD_IEEE80211 and other ARPHRD_IEEE80211_* constants are reserved for monitor-mode interfaces, again I guess because they use the raw 802.11 frame format.

Why is socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_ROUTE) a problem?

Looked again and you're correct; if all they are doing is enumerating interfaces you should be fine with that particular call.

ARP has nothing to do with this, the "ARP" in the names of ARPHRD_* constants is historical I think.

Yes that is why I said as a separate matter. It is not related to your fail.

I guess because you just send/receive regular ethernet frames on them

You don't send link layer frames of any type on WSL. I assumed NFD was going to try to do so, since it checks. If it is not, then you can try editing the code and return InterfaceType::ETHERNET unilaterally and see what happens (I have not).

I enabled WSL on my laptop for the lulz and it reports link/ieee802.11 as well. I suspect (but can't prove) WSL simply parrots the link encap that the underlying Windows driver reports. [The less likely but possible alternative being WSL sticks ARPHRD_IEEE80211 in there on it's own just because it's a wireless interface and that seemed like a good idea at the time.] It seems from the response you got NFD deliberately ignores such interfaces (as well as say ARPHRD_TUNNEL) by design.

WSL is doing a conversion from interface type to the ARPHRD_* type, but the conversion logic is essentially mnemonic (ethernetCsmacd->ARPHRD_ETHER, softwareLoopback->ARPHRD_LOOPBACK, ieee80211->ARPHRD_IEEEE80211, et al.). If Linux always reports ARPHRD_ETHER for all ethernet based interfaces then we can do the same.

From a quick look at linux kernel sources, ARPHRD_IEEEE80211 is in fact only used for monitor mode in some very old wifi drivers. Modern drivers (mac80211-based) use ARPHRD_ETHER in all interface modes except for monitor mode (NL80211_IFTYPE_MONITOR) where ARPHRD_IEEE80211_RADIOTAP is used.

This should be addressed in the Skip-ahead 17627 build.


Use SQLDelight to cache feeds

Kind (Required)

Improvement

Overview (Required)

Currently this app only holds the cache in the ViewModel. I want to cache using DB.

Links

https://github.com/cashapp/sqldelight

I wanna try 🙋

Thanks! Assigned 👍


feat: Add no_std support

Follow-on from https://github.com/rust-bakery/nom-derive/pull/33.
Adds no_std support, and adjusts GitHub actions workflow to check these cases as well.
Tested against thumbv7em-none-eabi target.

@chifflier when you've time, I think this is ready for review please.


An error occurred while installing script [...]

OS: Windows 10
Command:
docker run -p 8800:8800 -v /l/_MEDIA/TorrentOut:/downloads  -v /c/Users/Hannah/Git/Docker/seedsync/config:/config ipsingh06/seedsync

Debug Logs:
2019/04/24 09:59:52 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2019/04/24 10:00:12 - DEBUG - seedsync - Persisting states to file
2019/04/24 10:00:23 - DEBUG - seedsync.ControllerJob - Thread ControllerJob started
2019/04/24 10:00:23 - DEBUG - seedsync.ControllerJob - Calling setup for ControllerJob
2019/04/24 10:00:23 - DEBUG - seedsync.Controller - Starting controller
2019/04/24 10:00:23 - DEBUG - seedsync.WebAppJob - Thread WebAppJob started
2019/04/24 10:00:23 - DEBUG - seedsync.WebAppJob - Calling setup for WebAppJob
2019/04/24 10:00:23 - DEBUG - seedsync.WebAppJob - Finished setup for WebAppJob
2019/04/24 10:00:24 - DEBUG - seedsync.Controller.MPLogger - Started listener thread
2019/04/24 10:00:24 - DEBUG - seedsync.ControllerJob - Finished setup for ControllerJob
2019/04/24 10:00:24 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 199, in process
self.__propagate_exceptions()
File "/app/python/controller/controller.py", line 523, in __propagate_exceptions
self.__local_scan_process.propagate_exception()
File "/app/python/common/app_process.py", line 124, in propagate_exception
raise exc.re_raise()
File "/app/python/common/app_process.py", line 30, in re_raise
raise self.ee.with_traceback(self.tb)
File "/app/python/common/app_process.py", line 87, in run
self.run_loop()
File "/app/python/controller/scan/scanner_process.py", line 71, in run_loop
files = self.__scanner.scan()
File "/app/python/controller/scan/local_scanner.py", line 31, in scan
raise AppError(Localization.Error.LOCAL_SERVER_SCAN)
common.error.AppError: An error occurred while scanning the local system.
2019/04/24 10:00:24 - DEBUG - seedsync.ControllerJob - Calling cleanup for ControllerJob
2019/04/24 10:00:24 - DEBUG - seedsync.Controller - Exiting controller
2019/04/24 10:00:23 - DEBUG - ActiveScanner - Started process
2019/04/24 10:00:23 - DEBUG - RemoteScanner - Started process
2019/04/24 10:00:23 - DEBUG - RemoteScanner - Running a scan
2019/04/24 10:00:23 - DEBUG - LocalScanner - Started process
2019/04/24 10:00:24 - DEBUG - LocalScanner - Running a scan
2019/04/24 10:00:24 - ERROR - LocalScanner.LocalScanner - Caught SystemScannerError
Traceback (most recent call last):
File "/app/python/controller/scan/local_scanner.py", line 28, in scan
result = self.__scanner.scan()
File "/app/python/system/scanner.py", line 81, in scan
raise SystemScannerError("Path does not exist: {}".format(self.path_to_scan))
system.scanner.SystemScannerError: Path does not exist: /c/Users/Hannah/Downloads
2019/04/24 10:00:24 - DEBUG - LocalScanner - Process caught an exception
2019/04/24 10:00:24 - DEBUG - RemoteScanner.RemoteScanner - Local scanfs md5sum = 598ad3ef4659f41c8749d22c72f7a0fe
2019/04/24 10:00:24 - DEBUG - RemoteScanner.RemoteScanner.Sshcp - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no<EMAIL_ADDRESS>"echo '598ad3ef4659f41c8749d22c72f7a0fe /tmp/scanfs' | md5sum -c --quiet"
2019/04/24 10:00:24 - DEBUG - ExtractProcess - Started process
2019/04/24 10:00:24 - DEBUG - ExtractDispatch - Started worker thread
2019/04/24 10:00:24 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 199, in process
self.__propagate_exceptions()
File "/app/python/controller/controller.py", line 523, in __propagate_exceptions
self.__local_scan_process.propagate_exception()
File "/app/python/common/app_process.py", line 124, in propagate_exception
raise exc.re_raise()
File "/app/python/common/app_process.py", line 30, in re_raise
raise self.ee.with_traceback(self.tb)
File "/app/python/common/app_process.py", line 87, in run
self.run_loop()
File "/app/python/controller/scan/scanner_process.py", line 71, in run_loop
files = self.__scanner.scan()
File "/app/python/controller/scan/local_scanner.py", line 31, in scan
raise AppError(Localization.Error.LOCAL_SERVER_SCAN)
common.error.AppError: An error occurred while scanning the local system.
2019/04/24 10:00:24 - DEBUG - ActiveScanner - Process received terminate flag
2019/04/24 10:00:24 - DEBUG - ActiveScanner - Exiting process
2019/04/24 10:00:26 - DEBUG - ExtractProcess - Process received terminate flag
2019/04/24 10:00:26 - DEBUG - seedsync.Controller.MPLogger - Stopped listener thread
2019/04/24 10:00:26 - INFO - seedsync.Controller - Exited controller
2019/04/24 10:00:26 - DEBUG - seedsync.ControllerJob - Finished cleanup for ControllerJob
2019/04/24 10:00:26 - DEBUG - seedsync.ControllerJob - Thread ControllerJob stopped
2019/04/24 10:00:26 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener

Progress. Tried a different server and now it's just perpetually waiting for remote.

2019/04/24 11:22:20 - ERROR - RemoteScanner.RemoteScanner - Caught scp exception
Traceback (most recent call last):
File "/app/python/controller/scan/remote_scanner.py", line 115, in _install_scanfs
remote_path=self.__remote_path_to_scan_script)
File "/app/python/ssh/sshcp.py", line 169, in copy
args=" ".join(args)
File "/app/python/ssh/sshcp.py", line 109, in __run_command
raise SshcpError(sp.before.decode())
ssh.sshcp.SshcpError:
scp: /home25/haboutnnah/bin/seedsync/scanfs: No such file or directory
2019/04/24 11:22:20 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 199, in process
self.__propagate_exceptions()
File "/app/python/controller/controller.py", line 524, in __propagate_exceptions
self.__remote_scan_process.propagate_exception()
File "/app/python/common/app_process.py", line 124, in propagate_exception
raise exc.re_raise()
File "/app/python/common/app_process.py", line 30, in re_raise
raise self.ee.with_traceback(self.tb)
File "/app/python/common/app_process.py", line 87, in run
self.run_loop()
File "/app/python/controller/scan/scanner_process.py", line 71, in run_loop
files = self.__scanner.scan()
File "/app/python/controller/scan/remote_scanner.py", line 57, in scan
self._install_scanfs()
File "/app/python/controller/scan/remote_scanner.py", line 118, in _install_scanfs
raise AppError(Localization.Error.REMOTE_SERVER_INSTALL)
common.error.AppError: An error occurred while installing scanner script to remote server.
2019/04/24 11:22:20 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 199, in process
self.__propagate_exceptions()
File "/app/python/controller/controller.py", line 524, in __propagate_exceptions
self.__remote_scan_process.propagate_exception()
File "/app/python/common/app_process.py", line 124, in propagate_exception
raise exc.re_raise()
File "/app/python/common/app_process.py", line 30, in re_raise
raise self.ee.with_traceback(self.tb)
File "/app/python/common/app_process.py", line 87, in run
self.run_loop()
File "/app/python/controller/scan/scanner_process.py", line 71, in run_loop
files = self.__scanner.scan()
File "/app/python/controller/scan/remote_scanner.py", line 57, in scan
self._install_scanfs()
File "/app/python/controller/scan/remote_scanner.py", line 118, in _install_scanfs
raise AppError(Localization.Error.REMOTE_SERVER_INSTALL)
common.error.AppError: An error occurred while installing scanner script to remote server.
2019/04/24 11:22:22 - INFO - seedsync.Controller - Exited controller

Please try the latest version and see if you still get this error.


PHP SDK status

I see that the PHP SDK is no longer actively maintained and has not seen a release in two years. Is there any plan for that SDK to get maintenance attention, or is it pretty much dead?

We're looking for new maintainers. Until that time the SDK remains inactive and will not receive updates.


Just does not work

cat ~/.s3cfg
[default]
access_key = XXXXXX-YYYYYYYYYYYYY
secret_key = tPVsgEQuKnVjknvr6dBvieVrsJK2afudeh1-GGu5
bucket_location = ru-central1
host_base = storage.yandexcloud.net
host_bucket = %(bucket)s.storage.yandexcloud.net
website_endpoint = http://%(bucket)s.website.yandexcloud.net

s3cmd works. s4cmd:
  (D)s4cmd:662  read S3 keys from $HOME/.s3cfg file
  (D)s4cmd:135  >> ls_handler(<__main__.CommandHandler object at 0x7f0966afc650>, ['ls'])
  (D)s4cmd:135  >> list_buckets(<__main__.S3Handler object at 0x7f0966afc5d0>)
  (D)s4cmd:405  >> S3APICALL list_buckets()
  (E)s4cmd:189  [Runtime Exception] An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.
  (E)s4cmd:191  Traceback (most recent call last):
  File "/usr/bin/s4cmd", line 1878, in <module>
    CommandHandler(opt).run(args)
  File "/usr/bin/s4cmd", line 1533, in run
    CommandHandler.__dict__[cmd + '_handler'](self, args)
  File "/usr/bin/s4cmd", line 136, in wrapper
    ret = func(*args, **kargs)
  File "/usr/bin/s4cmd", line 1604, in ls_handler
    self.pretty_print(self.s3handler().list_buckets())
  File "/usr/bin/s4cmd", line 136, in wrapper
    ret = func(*args, **kargs)
  File "/usr/bin/s4cmd", line 698, in list_buckets
    for bucket in self.s3.list_buckets().get('Buckets') or []:
  File "/usr/bin/s4cmd", line 406, in wrapped_method
    ret = getattr(self.client, method)(*args, **merged_kargs)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 324, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 622, in _make_api_call
    raise error_class(parsed_response, operation_name)
ClientError: An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.

s4cmd 2.0.1+ds-1

Make sure to define the endpoint_url too with your commands.


Future add-on? draft/standard unknown (EMOJI)

Right, so, taking IRC to the next lvl is not easy. Modern tech, is pushing hardware, and vice-versa. Fact, that people are aware/used to emojis, and since IRC(Oragono) is fully capable of supporting these,
any toughts about this?
Emoji
What this is atm: Actual language help
could become
this : Modern look for language help
I'm not sure if this should be even present, here.. It sounds more like a IRCv3+ stuff to implement (is it?).
But in any case, it would be an awesome thing to support.
L.E.  Should this be implemented server-side?  client-side?
Is there any true need for using emoji(except for "beautify" text chat?) [Most people would be happy, afaik]
Any drawbacks for using them? Any implementation problems? Client side OS/libs deps?
More L.E.  on Oragono ircd, all emoji codes work, in HexChat, but part of them, in WeeChat.

We could implement this without an issue. Just comes down to adding emoji flags to each language code, which isn't a problem and I could do no worries. The new method would be something like this though, since the language codes need to be conveyed:
🇦🇺 Australian English (en-AU)

Looking back at this, I believe the suggestion's more about using emoji more generally around the server. Could be cool for sure, and definitely worth us looking at. For example, putting some in the default MOTD (just as we have the colour charts), that sort of thing. Shouldn't be any particular protocol concerns, I don't think we really could put them anywhere that could mess clients up too much.

Well, some places would be:

available language list (each language with it's own flag);
some exclamation/question/info marks at the beginning of notice/global messages (maybe even coloured, like a red alert sign, for some important stuff?);
the same signs as ^, in MOTD, to point out some stuff;

As for the rest, like smilies & such, that's up to the users/clients.


RDCC-1087 : Removed regex validation for first name, last name

JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/RDCC-1087
Change description
Regex Validation for entering first and last name for User Profile PUT API to be reverted
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[X] No

https://rd-user-profile-api-pr-310.service.core-compute-preview.internal
Above PR is tested by QA and all looks OK. Screenshots attached for evidences in RDCC-1087
Now ready for merging into master.


Enforce integer for Nominal Magnification

Attribute to Validate
Specify the attribute that you would like validation rule to apply to
Imaging Level 2:
Nominal Magnification:
Validation Rule
Specify the validation rule, please refer to this document in the Schema Hub
int
How important is this validation? Select from the options below:
• Low - it's an enhancement but not crucial for work
• Medium - can do work without it; but it's important (e.g. to save time or for convenience)
• Important - it's a blocker and can't do work without it
When will use cases depending on this become relevant? Select from the options below:
• Short-term - 2-4 weeks
• Mid-term - 2-4 months
• ~~Long-term - 6 months - 1 year``
Additional context
Add any other context or screenshots about the validation rule here.
Currently some users add X as a suffix, eg 20X. The attribute description suggests this should be an integer, which would be a good way to enforce consistency.
Alternatively a regex could be used to enforce a number followed by the suffix X if preferable

Fixed in #99


bump JSON Schema test suite

as per #107
doing this separately from #108 because there are failing tests in draft3 and draft4 i.e. jesse's current implementation fails some "corner" cases
WORK IN PROGRESS

[x] draft3 - additionalProperties
[x] draft3 - ref
[x] draft4 - additionalItems
[x] draft4 - additionalProperties
[x] draft4 - enum
[ ] draft4 - ref
[x] draft4 - refRemote

FYI, I'm working on it

Have some difficulties with ref test case "Description: ref overrides any sibling keywords". We are currently crashing when we see that there are other validators siblings to $ref. But it seems that the test is supposed to just ignore all the other keys if there is a $ref defined. I have troubles trying to find a place where this logic can be hooked...

:wave:

Let's say that I'll merge it this Wednesday 8th of December, if there will be no objections until then =)


Request: Add Constrain window option.

Description of the new feature / enhancement
Prevent any edge of a window from leaving the screen.
(Low priority) With additional option to constrain to one monitor or to span across multiple monitors providing that they are operating at the same orientation, resolution and pixel density.
Scenario when this would be used?
Any time when I have a window which does not conform to any Fancy Zone,
Could combine as an option within Always on Top.
Supporting information
No response

I also have this issue - If using a custom fancyzone template (or a predefined one) - My right hand side window is always slightly outside of the screen bounds - which makes it pretty annoying!

This should be listed as a Fancy Zones bug, as windows in a Fancy Zone
which has an edge on the edge of the screen do spill outside of the screen,
onto the next screen if using a dual monitor setup.
On Tue, Jan 11, 2022 at 6:31 PM helloitsbirdo @.***>
wrote:

I also have this issue - If using a custom fancyzone template (or a
predefined one) - My right hand side window is always slightly outside of
the screen bounds - which makes it pretty annoying!
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/15445#issuecomment-1010460775,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXHS5XYNVZNSCUD5NFTCGQTUVS4TTANCNFSM5LWQWRFA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

I'm not using a dual monitor ser, just an ultrawide

Same issue, just doubly annoying, which I believe is related to window
edges are defined by their visible edge.
On Wed, Jan 12, 2022 at 4:48 AM helloitsbirdo @.***>
wrote:

I'm not using a dual monitor ser, just an ultrawide
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/15445#issuecomment-1010848674,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXHS5XYJCSKVJKSDRPHJWJDUVVE6NANCNFSM5LWQWRFA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>