Identify and address Google search and Dataset search issues

Being comprehensively indexed by search engines such as Google is a substantial benefit for DataONE and DataONE Members. Ideally, the whole variety of information (datasets, people, portals, metrics, etc.) housed within DataONE would be findable through traditional search engines and, for datasets, also Google Dataset Search.
As of 2021, our primary tool for knowing whether or not we are comprehensively indexed is the Google Search Console which provides a whole suite of tools for diagnosing issues.
Some of the problems we've addressed in the past include:

Improve MetacatUI's mobile support (https://github.com/NCEAS/metacatui/issues?q=is%3Aissue+is%3Aclosed+mobile)

This is important because Googlebot uses a mobile UA and penalized non-mobile-friendly pages

We now publish sitemaps on the CNs and MNs https://github.com/NCEAS/metacat/issues/1432 https://github.com/NCEAS/metacat/issues/1434 https://github.com/NCEAS/metacat/issues/1369 https://github.com/NCEAS/metacat/issues/1368 https://github.com/NCEAS/metacat/issues/1263

This is important because, while crawlers implement whichever methods they wish, some (like Google) use sitemaps to discover and define the set of [canonical](https://developers.google.com/search/docs/advanced/crawling/consolidate-duplicate-urls URLs for a site

Issues we have ahead of us include:

There's a separation between how we're doing SEO on dataone.org and search.dataone.org. We should probably integrate the two under dataone.org
For specific types of content (eg portals), our search index presence is very far from complete. Portal users would find it very valuable to show up in Google search. We need to find a way to include all of this stuff, not just datasets
Our dataset URLs aren't being fully indexed (see below)

Dataset coverage
Summary
At the time of writing, we have 846,622 dataset and portal URLs listed in our sitemaps and Google has discovered them all correctly. 812k of these are marked as "Excluded". When we drill down into the index coverage of those URLs, we get this breakdown:

Type
# Pages

Discovered - not currently indexed
770,302

Duplicate, submitted URL not selected as canonical
30,977

Duplicate without user-selected canonical
12,265

Crawled - currently not indexed
7,986

Pages with redirect
424

Blocked due to other 4xx issue
6

Discovered - not currently indexed
The majority of these are "Discovered - not currently indexed", which his defined as:

Discovered - currently not indexed: The page was found by Google, but not crawled yet. Typically, Google wanted to crawl the URL but this was expected to overload the site; therefore Google rescheduled the crawl. This is why the last crawl date is empty on the report.

This makes sense to me as our individual dataset landing pages are very slow relative to what Google expects. I'm hoping that we can hear more back from the Google team about whether this is truly what's going on or if it's something else. We know from unofficial sources that Google's crawl infrastructure has two queues: One for fast sites, and one for sites it had hoped were fast but put in a separate, lower-priority queue. I'd guess we're in the latter.
Duplicate, submitted URL not selected as canonical
What we see here is that Google is selecting a URL like https://dataone.org/datasets/R1-x138-079-0042-010 as our canonical. These URLs are reported in our embedded JSON-LD and they really are our canonical URLs.  I think we should consider switching our sitemap implementation on the CNs to use https://dataone.org/datasets URLs instead.
Other things
Aside from the above categories, some of the odd stuff we've seen is:

Some URLs are listed as not present in sitemaps even when they are. e.g.,

https://search.dataone.org/view/0aa7877f6fe1de79bd0d71a54115f9e5
https://search.dataone.org/view/ess-dive-c9503304a84ea68-20201029T231811168

Google sets a different URL as canonical

Good case: I found one example where a user broke their version chain somehow and Google actually fixed it.
Bad case: Google picks https://search.dataone.org/signin as a canonical for some URLs. Not sure what's up with this

[Note: Please feel to edit this issue to be more complete]

Thanks for the summary, Bryce.
One thing we can do right off is to eliminate crawling of our stage, dev, and sandbox hosts, to eliminate some of the errors we see from those hosts. I suggest that we add:
    ## Disallow Robots for all content
    Header Set X-Robots-Tag "noindex, nofollow"

to the apache config for all non-production search and API servers. This is the google recommended way to indicate robot preferences (instead of robots.txt), and it is really easy to implement. I tested it on cn-stage.test.dataone.org and it seems to deliver the expected headers.

Thanks @mbjones, that sounds like a good step.
As an update about our coverage in Google Search, since writing this issue and talking with folks at Google, our coverage has gone up a bit, by maybe 10k records. So we can track progress here, I'm counting 48,434 sitemap entries covered, 812k excluded.

I also see a similar exclusion pattern for the Arctic Data Center (18.3K Valid, 11.6K Excluded). The following is the breakdown of the excluded URIs by reason on the ADC site:

Just updating on status here: I've had a few back and forths with folks at Google and they're planning to take a closer look at things. The vast majority of our dataset landing pages (~660k) aren't indexed at all, despite being discovered by Google via our sitemaps.
We have at least one ticket on this topic but I can't seem to find it right now.
I'll update here when I hear back from Google.

Ran into a new twist today. It looks like a Dataset record will fail Google's validation if it doesn't have a description between 50 and 5000 characters. @mbjones found this out while looking at some the invalid datasets in our Search Console and checking Google's guidelines.
@mbjones said on Slack:

I pulled our abstracts from SOLR for all non-obsoleted metadata in DataONE; we have 857 datasets with abstract > 5000 chars, and 434,422 with abstract < 50 chars. Of the latter , 409695 are missing abstracts

So about half of DataONE's content may be failing validation for this reason. Adding this note here just for the papertrail.

Can we create a description from other parts of the metadata if there is no abstract? Such as piece together the creator names, title, location, etc. to create a pseudo abstract?


Upgrade to Python 3.8 for hello program

Here's the last part of the stack trace I think time.clock isn't working for Python 3.8
File "c:\users\rskov\appdata\local\programs\python\python38\lib\site-packages\mako\compat.py", line 145, in
time_func = time.clock
AttributeError: module 'time' has no attribute 'clock'

Looks like you were wrong repo. This repo is about examples in flask. In these examples mako is not used.

Thank you!

Thank you!


[async] Figure out flashing/reloading of recent data, perhaps lack of caching?

🧰 Task
If you open a multiscale image (in this case a list of dask arrays) and then zoom in and out you will observe that as you zoom out the data is reloaded: the indicator shows loading and there is a flashing effect.
https://github.com/user-attachments/assets/3682d776-d141-4c0a-9c6e-cd05992ae3d5
But it happens with simple numpy as well:
import numpy as np
array = np.random.random((10_000, 10_000))
viewer.add_image([array, array[::2, ::2], array[::4, ::4], array[::8, ::8]])

Since we looked at those tiles already, I would expect them to be cached? Or is this the effect of passing data to GPU?

They should be cached by default, but there's nothing in the current async implementation that will bypass the async call if the data is in the cache - i.e. it always makes the call async. If other parts of slicing or pushing the data to the GPU takes some non-negligible time, then I'd expect to see this kind of behavior.
I think the flashing while zooming out is more related to the way slicing/rendering is done (i.e. by pushing the new field-of-view as a new texture to the GPU, rather than updating sub-regions of existing textures). There are some potentially easy workarounds (e.g. fetching more than the current field of view), but a better longer term solution is needed.

Thanks for the insight @andy-sweet !


Make syncChunkWrites false by default

What config to change
syncChunkWrites=true -> syncChunkWrites=false via the vanilla's options.txt file.
Why should it be changed
By default this feature causes region files to be opened in synchronous mode and is only enabled in Windows since 1.16 Pre-Release 3, However it has some unintended side effects such as causing world savings to become horribly slower than expected which is apparent in certain Linux distros when exiting minecraft servers using the stop command.
Additional details
While this doesn't really change anything in terms of optimizations (or might for some?), it does resolve some annoyances when dealing with backend lags in this case. However this is disabled by default on many servers such as PaperMC which is the reasoning why i thought of making this issue in first place.

I'm confused. The wiki and your post says it is enabled by default on Windows, but you mention that "world savings become horribly slower [...] in certain Linux distros [...]".
So what does it have to do with Windows?

I'm confused. The wiki and your post says it is enabled by default only on Windows, but you mention that "world savings become horribly slower [...] in certain Linux distros [...]".
So what does Linux distro slowness have to do with Windows?

This option is actually the default on vanilla servers which means it's still bad to leave it enabled, but since managing region files in synchronous mode is absolute crap regardless of OS - pretty much should not be enabled at all in case any more performance loss happen.

This option is actually the default on vanilla servers [regardless of the OS]

So... it is a server problem, if it is default enabled on there but not on all clients?
I feel like there is not really a reason for FO to touch it, considering Mojang did not add any UI option to touch it, and because FO has FastQuit anyway.

This option is actually the default on vanilla servers [regardless of the OS]

So... it is a server problem, if it is default enabled on there but not on all clients?
I feel like there is not really a reason for FO to touch it, considering Mojang did not add any UI option to touch it, and because FO has FastQuit anyway.

FastQuit does not solve the problem when the value was set to true, therefore this is not an ideal workaround for that.

Decided against this, do not want to risk it.


Add support for picking render adapter

This PR adds support for selecting the render adapter through a file located at C:\IddSampleDriver\adapter.txt.
The feature is useful for devices with multiple GPUs (ie. laptops).
The behaviour is pretty straightforward. If the adapter specified is not found then it behaves the same way as it did before and leaves it up to the OS to select an adapter. And likewise if the running version of Windows doesn't support the API call.
Sample file:
adapter.txt

You're a lifesaver dude 🙏
This is EXACTLY what I needed on my Acer Predator, which has a very capable GPU letting me play a decent amount of games that support HDR out and could handle streaming them fine, but unfortunately the default behavior of this driver forced the adapter to be my igpu which doesn't support HEVC encoding. Now it works like a charm!!
Highly recommend this get merged ASAP, I checked around and quite a few people with laptop GPUs were hoping to use this for HDR streaming but were unable to.
I did manage to stream HDR before this PR by disabling my igpu driver, which isn't ideal cuz that forced my nvidia to render both screens or i had to force the main monitor to not display anything. Plus I wasn't able to control the backlight of the monitor anymore. Glad I don't have to do that anymore!

Sample file:

After looking at it, how would I write another card, is it the friendly name of the gpu? What if you have multiples of the same card in the machine, how would you be able to chose which?

It's just the friendly name of the adapter as it seemed easier to use during configuration.
Using Vendor ID and Device ID would probably be better for more specific selection. Maybe we can go with that instead?

As for multiple cards of the same model, I think Windows puts a number at the end of the adapter name to distinguish them if it sees same name. So the name method is probably fine.

As for multiple cards of the same model, I think Windows puts a number at the end of the adapter name to distinguish them if it sees same name. So the name method is probably fine.

Thanks for your work on this! Testing and pushing ASAP. :)

This has been highly requested. Huge thanks for this one. People are going to be very excited. Once it's out, I'll make an update video and post with credit and such showing the functionality you've added. :)

I can test the multiple gpu with the same gpu if needed. I dont mind. Although we definitely might need a different solution to selecting the render device. Its hard to do it without the companion app. Although if we set it with vendor and device id it may not be as easy for the unexperienced


[grid] Make storybook work again.

Missed this previously ☮

You really need to alphabetize your sources man.

👍

I hang my head in the utmost shame

Also was that meant to be a Make America Great Again reference? NOT FUNNY BROOOOO
😢 😢 😢

NOOOooooo. I did accidentally use that format once… it’s amazing how brainwashed you get from a few weeks of daily news in the US.

Pretty sad that X can never do Y again without all this baggage attached to it....

Make making something great great again!


[Question] Any support for Pixel-exclusive features and/or LSposed+Pixelify?

I am currently using GrapheneOS rooted with AVBroot, and one of my main goals with rooting was to reimplement some PixelOS-exclusive features using a module like Pixelify. I was disappointed to discover that Pixelify, as well as basically every LSposed module I tried to install, resulted in a boot loop.
Have you tried getting any Pixel features (like Call Screening, Hold For Me, pretty much anything involving Android System Intelligence) to work? Have you had any success?

Hi there! thanks for reaching out!
Official Magisk by Topjohnwu is completely broken on GrapheneOS. Last time I tried Magisk, I had no root access (except in shell, yes) and no Zygisk. That's why I made a fork that fixes the root and Zygisk on GrapheneOS.
From what I remember, modules like Pixelify requires you to have LSPosed installed(?). This I believe will not work because:

LSPosed does not work in GrapheneOS
Pixelify requires LSPosed to work
Pixelify requires certain apps to be system apps with necessary permissions granted to it (GrapheneOS is all about giving apps the least amount of privilege; even for the user) and might result in device getting bricked if tried when bootloader is locked

Magisk devs are hostile towards GrapheneOS and its users. And hence you can expect little to no response from their end for anything related to GrapheneOS.

Have you tried getting any Pixel features (like Call Screening, Hold For Me, pretty much anything involving Android System Intelligence) to work? Have you had any success?

I did try stuffs back then and none of them worked. Gave up!

At the moment of me writing this, I have got no time (at least, until Feb 2025) to work neither on this project nor test the OS on my phone. But hey, it is pretty stable and is known to work!
The project's README will give you a brief idea about what the additionals that this project injects while signing OTA with your own keys.

Thanks for your answer! By the way, did you ever get mGMScompat to be functional, or is it archived because it's hopeless? I too believe it would be incredibly useful to sandbox google apps on any ROM with just a magisk module, but it looks like maybe the project was unsuccessful? What's up with that?

i bought a pixel and never found the need to develop mGMSCompat. So I archived the project before it even started.
and yes, it is highly complicated than thought. it will do more harm than good. it requires zygisk to be functional to modify source code (about 5k+ lines of code needs to be modified) in run time. active maintenance is a must given how grapheneos is changing.

Understandable, thank you. I also have a Pixel with GrapheneOS, but I miss a lot of the Pixel-exclusive features (which I had asked about when I opened this issue, lol). I was hoping that mGMScompat would allow me to use stock PixelOS with root and still be able to sandbox GMS. Oh well. It's a monumental project and I'm not surprised it proved unfeasible.

No. mGMSCompat requires user to have an AOSP based OS like LineageOS. It cannot be used on StockOS.
Also, in order to get mGMSCompat working on StockOS, you would have to completely remove GMS from StockOS which might remove all the Pixel Specific features


added chatbot Integration

Change description
Integrated AWS chatbot service for manual approval process in the code pipeline
Type of change

[ √] New feature (adds functionality)

Related issues

Fix #1

Checklists
Development

[ ] Lint rules pass locally
[ ] Application changes have been tested thoroughly
[ ] Automated tests covering modified code pass

Security

[ ] Security impact of change has been considered
[ ] Code follows company security practices and guidelines

Code review

[ ] Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
[ ] "Ready for review" label attached and reviewers assigned
[ ] Changes have been reviewed by at least one other contributor
[ ] Pull request linked to task tracker where applicable

@maxr-sg I think it looks solid now, I added some default values to prevent breaking changes. I do think we need some sort of README docs in the TF docs generator file that states on how to configure Slack.


[HOTFIX] Remove spark dependency in Processing module

Why is this PR needed?
Remove spark dependency in Processing module, as it is not required
What changes were proposed in this PR?
Removed spark dependency in Processing module
Does this PR introduce any user interface change?

No

Is any new testcase added?

No

Build Success with Spark 2.4.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbon_PR_Builder_2.4.4/562/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/2263/


Vidble albums fails to rip

Attempting to rip vidble albums gives an error:

Failed to load https://www.vidble.com/album/xxxxxxxx after 1 attempts

(Replace xxxxxxxx with album ID)

It appears to be related to an HTTPS certificate issue with vidble. Ripping from the same URL works with http:// but not https://
Attempting to save a file from vidble using wget gives:

Resolving www.vidble.com (www.vidble.com)... <IP_ADDRESS>
Connecting to www.vidble.com (www.vidble.com)|<IP_ADDRESS>|:443... connected.
ERROR: The certificate of ‘www.vidble.com’ is not trusted.
ERROR: The certificate of ‘www.vidble.com’ hasn't got a known issuer.

So, a workaround might be to use the regular http connection. Unfortunately, when you rip a user profile from reddit, vidble links default to https.

ERROR: The certificate of ‘www.vidble.com’ is not trusted.

ERROR: The certificate of ‘www.vidble.com’ hasn't got a known issuer.

This might be an issue on your end. COMODO CA Limited (The issuer for vidble) is trusted on my PC.
Can you post a test link?

Testing with this link https://www.vidble.com/album/sQEMXFue
ripme seems to work fine even with https, so it just seems that COMODO CA Limited isn't trusted on your computer

Web page loads fine for me but the ripper fails with something resembling a cert error. I'll try it again after updating java, but cert authorities have breaches and loss of trust all the time, and various root certs remove trust in them.
Here's the log from the console (more detailed than the one in the GUI): log.txt
Anyway what if we forced http for this site and just ignored the https cert altogether?

In fact, we don't force https, so if you use an http URL, the site rips fine:
>java -jar ripme.jar -u http://www.vidble.com/album/sQEMXFue
Loaded C:\dev\ripme\rip.properties
Loaded log4j.properties
Initialized ripme v1.4.19
Retrieving http://www.vidble.com/album/sQEMXFue
    Downloading file: http://www.vidble.com/ktFsjTqAbu.jpg Retry #1
    Downloading file: http://www.vidble.com/3BB3trFRXd.jpg Retry #1
    Downloading file: http://www.vidble.com/Hm4q0sq4ir.jpg Retry #1
    Downloading file: http://www.vidble.com/7MveFlkDaZ.jpg Retry #1
    Downloading file: http://www.vidble.com/JNNI4yVExF.jpg Retry #1
[+] Saved http://www.vidble.com/ktFsjTqAbu.jpg as .\rips\vidble_sQEMXFue\001_ktFsjTqAbu.jpg
    Downloading file: http://www.vidble.com/nIN9B3jh8T.jpg Retry #1
[+] Saved http://www.vidble.com/nIN9B3jh8T.jpg as .\rips\vidble_sQEMXFue\006_nIN9B3jh8T.jpg
[+] Saved http://www.vidble.com/JNNI4yVExF.jpg as .\rips\vidble_sQEMXFue\005_JNNI4yVExF.jpg
[+] Saved http://www.vidble.com/7MveFlkDaZ.jpg as .\rips\vidble_sQEMXFue\004_7MveFlkDaZ.jpg
[+] Saved http://www.vidble.com/3BB3trFRXd.jpg as .\rips\vidble_sQEMXFue\003_3BB3trFRXd.jpg
[+] Saved http://www.vidble.com/Hm4q0sq4ir.jpg as .\rips\vidble_sQEMXFue\002_Hm4q0sq4ir.jpg

I found a workaround for this issue.  You have to import the Vidble certifcate into your keystore where java runs.

Open https://vidble.com in your browser.  Click on the lock sign and export the certificate.  Should be named 'vidblecom.crt'

Run 'keytool -import -noprompt -trustcacerts -alias vidblecom -file vidblecom.crt -keystore /etc/ssl/certs/java/cacerts'

Where 'cacerts' is the keystore on your system.
Password is 'changeit'

Using http links is also a workaround, and a fair bit simpler. :)

Anyway, this is not something ripme can fix directly. There are multiple workarounds in this thread. Closing this issue.


Fast Minimap are now in Test

Discussed in https://github.com/Nasller/CodeGlancePro/discussions/88

Originally posted by Nasller August  9, 2023
Hello dear users, I am pleased to announce that Fast Minimap will be publicly tested. My personal abilities are limited and I need more people to join the testing to ensure its stability. Thank you again for your feedback
Preview channel

#92

#93

#94

#96

#104


Create L.Ellipse.v1

This is the updated version for Leaflet 1+
Sorry I couldn't figure out how to do the pull request.

Pulled updates into main code rather than a seperate file.


Toolkit crashes when double-clicking on a transaction

Name of Defective Feature
Toolkit crashes when double-clicking on a transaction
System Settings (Required)
ToolKit Version: 0.7.13
Operating System (Mac/Windows/Etc): Mac
Browser (Chrome/Firefox/Safari): Firefox
Browser Version: 54.0
Other Extensions:
Export of Toolkit Settings (Bottom Left of Settings Page)
[{"key":"accountsClearSelection","value":false},{"key":"accountsDisplayDensity","value":"0"},{"key":"accountsEmphasizedOutflows","value":false},{"key":"accountsRowHeight","value":"0"},{"key":"accountsStripedRows","value":true},{"key":"activityTransactionLink","value":true},{"key":"betterScrollbars","value":"0"},{"key":"budgetBalanceToZero","value":true},{"key":"budgetProgressBars","value":"0"},{"key":"budgetQuickSwitch","value":false},{"key":"budgetRowsHeight","value":"0"},{"key":"calendarFirstDay","value":"0"},{"key":"categoryActivityPopupWidth","value":"0"},{"key":"changeEnterBehavior","value":true},{"key":"checkCreditBalances","value":true},{"key":"checkNumbers","value":false},{"key":"closeReconcileWindow","value":true},{"key":"collapseExpandBudgetGroups","value":true},{"key":"collapseSideMenu","value":true},{"key":"colourBlindMode","value":false},{"key":"compactIncomeVsExpense","value":true},{"key":"daysOfBuffering","value":true},{"key":"daysOfBufferingHistoryLookup","value":"12"},{"key":"editButtonPosition","value":"0"},{"key":"enableRetroCalculator","value":true},{"key":"enlargeCategoriesDropdown","value":true},{"key":"goalIndicator","value":true},{"key":"goalIndicatorWarningColor","value":false},{"key":"googleFontsSelector","value":"0"},{"key":"hideAOM","value":true},{"key":"hideHelp","value":false},{"key":"currentMonthIndicator","value":true},{"key":"highlightNegativesNegative","value":false},{"key":"importNotification","value":"0"},{"key":"incomeFromLastMonth","value":"0"},{"key":"l10n","value":"0"},{"key":"largerClickableIcons","value":false},{"key":"monthlyNotesPopupWidth","value":"0"},{"key":"navDisplayDensity","value":"0"},{"key":"pacing","value":"0"},{"key":"popupCalculator","value":false},{"key":"printingImprovements","value":true},{"key":"privacyMode","value":"0"},{"key":"reconciledTextColor","value":"0"},{"key":"removePositiveHighlight","value":false},{"key":"removeZeroCategories","value":true},{"key":"resizeInspector","value":true},{"key":"rightClickToEdit","value":true},{"key":"runningBalance","value":true},{"key":"seamlessBudgetHeader","value":false},{"key":"showIntercom","value":true},{"key":"splitKeyboardShortcut","value":true},{"key":"splitTransactionAutoAdjust","value":true},{"key":"squareNegativeMode","value":false},{"key":"stealingFromNextMonth","value":true},{"key":"swapClearedFlagged","value":false},{"key":"toBeBudgetedWarning","value":false},{"key":"toggleSplits","value":true},{"key":"toggleTransactionFilters","value":"0"},{"key":"reports","value":true},{"key":"warnOnQuickBudget","value":false},{"key":"spareChange","value":false}]

Steps to Reproduce the Behavior
Please write down the steps necessary to reproduce teh behavior you're seeing. Please be as thorough as possible.

Go to account list
Double click on any transaction
Crashes with the message:

Uh-oh!
An error occurred in Toolkit for YNAB (a browser extension you installed to enhance YNAB). Click here to report the issue to the YNAB Toolkit project.
Expected Behavior
What did you expect would happen after completing the steps?
Show fields so that I can update memo
Actual Behavior
What actually happened after you completed the steps.
Crashes - see above

hey @MichaelNMaggs this is actually because the latest version of the toolkit is version 1.1.0 and firefox is still on version 0.7.13 for some unknown reason, firefox is not approving very many, if any, extensions right now we're currently number 22 of 183 in their queue.
The bug you're seeing is actually fixed in todays release but unfortunately, unless you switch to use Chrome or Firefox approves our extension, you'll continue to see this behavior.

Many thanks for the update!

@MichaelNMaggs I decided to be proactive and reached out to Firefox. v1.1.0 has now been approved and you should be able to use firefox! Let me know how that goes!

Thanks for talking to Firefox. I have just tried updating, and 1.1.0 has now installed.  The issue has gone away!


Identity factors

This is to fix #128

I've been working through multi.py and trying to make EmbeddedTensor support the Tensor operations used there. I'm looking at the reshape calls in multi_mv and multi_solve. Is it easy to explain how these are used? Is it just adding/removing dimensions of size 1? Does it ever even turn one dimension of size 6 into two dimensions of sizes 2 and 3, let alone something more nefarious? @davidweichiang

If there are nonterminals $X_1., .., X_n$, where $X_i$ has shape $(d_{i,1}, ..., d_{i,m_i})$, let $M = (\mathbb{R}^{d_{1,1}} \otimes \dots \otimes \mathbb{R}^{d_{1,m_1}}) \oplus \dots \oplus (\mathbb{R}^{d_{n,1}} \otimes \dots \otimes \mathbb{R}^{d_{n,m_n}})$. Then a MultiTensor is an element of $M$ or a linear transformation $M \rightarrow M$. The job of multi_mv(a,b) is to apply a to b, and the job of multi_solve is to solve the equation x = ax+b.
The purpose of the reshapes is to flatten one of the $\mathbb{R}^{d_{i,1}} \otimes \dots \otimes \mathbb{R}^{d_{i,m_i}}$ into a vector. So, it's not just squeezing/unsqueezing. Note that it's possible for $m_i = 0$, so this actually "flattens" a scalar into a 1-element vector.

Ok so if I understand you correctly, the granularity of each $d_{i,j}$ is that of a letter in an einsum equation, not finer but also not coarser. I should be able to preserve sparsity through a multi_mv operation...

Sorry I've been letting this sit so long. What would be needed to integrate this into the rest of the code? Are there particular things that you need me to check?

Hi! Aria is doing two things (digging a tunnel from both ends):

Write more tests for einsum
Replace Tensor by EmbeddedTensor in MultiTensor and see what breaks

Maybe you can make a quick start on #2? Maybe it will give rise to test failures that Aria can fix??

Compared to the :heavy_check_mark: changes on this identity branch, I'm a lot less confident about the changes I made in the multi_embedded branch to get the sum_product tests to even run. But on that branch, we are down to just one test failure now:
======================================================================
ERROR: test_autograd (test.test_sum_product.TestSumProduct.test_autograd) (example='test/simplefgg.json (p=1.0)')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ccshan/u/rational/fmitf/fggs/test/test_sum_product.py", line 108, in test_autograd
    self.assertTrue(torch.autograd.gradcheck(f, in_values, atol=1e-3))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ccshan/u/rational/fmitf/python/lib/python3.11/site-packages/torch/autograd/gradcheck.py", line 1418, in gradcheck
    return _gradcheck_helper(**args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ccshan/u/rational/fmitf/python/lib/python3.11/site-packages/torch/autograd/gradcheck.py", line 1432, in _gradcheck_helper
    _gradcheck_real_imag(gradcheck_fn, func, func_out, tupled_inputs, outputs, eps,
  File "/home/ccshan/u/rational/fmitf/python/lib/python3.11/site-packages/torch/autograd/gradcheck.py", line 1075, in _gradcheck_real_imag
    gradcheck_fn(func, func_out, tupled_inputs, outputs, eps,
  File "/home/ccshan/u/rational/fmitf/python/lib/python3.11/site-packages/torch/autograd/gradcheck.py", line 1131, in _slow_gradcheck
    raise GradcheckError(_get_notallclose_msg(a, n, i, j, complex_indices, test_imag))
torch.autograd.gradcheck.GradcheckError: Jacobian mismatch for output 0 with respect to input 1,
numerical:tensor([[0.]], dtype=torch.float64)
analytical:tensor([[1.]], dtype=torch.float64)

On a larger scale, I'm still wondering about the relationship of Tensor, Semiring, and EmbeddedTensor. Do we have it right? The fact that Semiring uses Tensors seems backwards, but it had to be that way because of the difficulty of subclassing Tensor to use other semirings. OTOH, EmbeddedTensor already goes through the trouble of (sort of) subclassing Tensor, which makes me wonder whether the relationship between EmbeddedTensor and Semiring could be put right. That is, maybe an EmbeddedTensor could contain a Semiring and a Tensor, so that when you, e.g., add two EmbeddedTensors, it looks up the addition operation from the Semiring.

Currently EmbeddedTensor(t) converts t:Tensor into an EmbeddedTensor, and et.to_dense() converts et:EmbeddedTensor into a Tensor. We can try to make these conversions more implicit, but what kind of "using FGGs but not PERPL" are we trying to ease? Currently, the sum_product and viterbi code only uses EmbeddedTensor internally (but in order to support product and sum types, I'm working to change that), so it's a good time to think about what the interface should be (should sum_product sometimes return an EmbeddedTensor?). It might make sense to make sum_product and viterbi operate on an abstract TensorLike type, like some Semiring methods do.
One reason to not put a Semiring inside (Embedded)Tensors is the binary method problem: what should t1+t2 do if t1 and t2 contain different Semirings?
If you could point out particular places in indices.py that are hard to understand, I'm sure there are plenty of comments to add...

Since the word "embedding" is already overused in a ML context, are there any other names you can think of? StructuredTensor?

Since the word "embedding" is already overused in a ML context, are there any other names you can think of? StructuredTensor?

How about "pattern"?

"embedding" -> "(sparsity) pattern"
"embedding var(iable)" -> "(sparsity-)pattern var(iable)"
"embedded tensor" -> "(sparsity-)patterned tensor"

That works for me.

That works for me.

Is it easy to explain why EmbeddingVar isn't just an int (0 for the first physical axis, 1 for the second physical axis, etc.)? And what Substs are used for?

Is it easy to explain why EmbeddingVar isn't just an int (0 for the first physical axis, 1 for the second physical axis, etc.)?

Is it enough to say that we want to distinguish between the first physical axis of one PatternedTensor and the first physical axis of another PatternTensor?

And what Substs are used for?

Unification is first and foremost used to implement einsum. For example to einsum "ij,jk->ik" (in other words matrix multiplication), we unify the second virtual axis of the first PatternedTensor against the first virtual axis of the second PatternedTensor, and that might add some entry to the Subst that maps some physical axis of the first PatternedTensor to some product of physical axes of the second PatternedTensor.

Hmm, why are my last 7 commits to the identity branch (ending with 89e838d) not shown in the pull request? Anyway, I think I have addressed all your comments.


python.pkgs.grpcio: use system openssl, zlib, and c-ares

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@GrahamcOfBorg build python.pkgs.grpcio python3.pkgs.grpcio python38.pkgs.grpcio

This seems to be broken in latest unstable:
/build/source/third_party/boringssl-with-bazel/linux-x86_64/crypto/chacha/chacha-x86_64.S:56: undefined reference to `OPENSSL_ia32cap_P'
/nix/store/iygzjy14i075hkxr2gq8cpfjvk2rkx0b-binutils-2.31.1/bin/ld: python_build/temp.linux-x86_64-3.8/third_party/boringssl-with-bazel/linux-x86_64/crypto/chacha/chacha-x86_64.o: relocation R_X86_64_PC32 against undefined hidden symbol `OPENSSL_ia32cap_P' can not be used when making a shared object
/nix/store/iygzjy14i075hkxr2gq8cpfjvk2rkx0b-binutils-2.31.1/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status

I opened https://github.com/NixOS/nixpkgs/issues/97811.


chore: Add Clarity to Manifests Installation Guide

Description of your changes:
Add steps to show installation of ODH operator and oc client in prerequisites
Environment tested:

Python Version (use python --version): N/A
Tekton Version (use tkn version): N/A
Kubernetes Version (use kubectl version):  1.25.4
OS (e.g. from /etc/os-release): N/A

Checklist:

[x] The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

/approve

/lgtm

/lgtm


The error when running the scafe.workflow.sc.pool

Hello, I meet another trouble. Hope you can help me. Thank you in advance.
I want to combine three samples into one pool. Firstly, I create my own lib_list_path.txt. It looks like this.
S17765-F11_S33  1       /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S33_SCAFE_out/bam_to_ctss/S17765-F11_S33/bed/S17765-F11_S33.UMI_CB.ctss.bed.gz    /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S17765-F11_S33_out/outs/filtered_feature_bc_matrix/barcodes.tsv.gz        /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S33_SCAFE_out/bam_to_ctss/S17765-F11_S33/bed/S17765-F11_S33.CB.ctss.bed.gz
S17765-F11_S34  2       /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S34_SCAFE_out/bam_to_ctss/S17765-F11_S34/bed/S17765-F11_S34.UMI_CB.ctss.bed.gz    /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S17765-F11_S34_out/outs/filtered_feature_bc_matrix/barcodes.tsv.gz        /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S34_SCAFE_out/bam_to_ctss/S17765-F11_S34/bed/S17765-F11_S34.CB.ctss.bed.gz
S17765-F11_S36  3       /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S36_SCAFE_out/bam_to_ctss/S17765-F11_S36/bed/S17765-F11_S36.UMI_CB.ctss.bed.gz    /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S36_SCAFE_out/bam_to_ctss/S17765-F11_S36/bed/S17765-F11_S36.CB.ctss.bed.gz        /storage/yhhuang/users/ruiyan/MS_Cambridge/S17765-F11/S17765-F11_S36_out/outs/filtered_feature_bc_matrix/barcodes.tsv.gz
Then I run the script as this
scafe.workflow.sc.pool --overwrite=yes --lib_list_path=./lib_list_path.txt --genome=hg38.gencode_v32 --run_tag=S17765-F11 --run_outDir=./sc.pool
However, it produces an error like this

But I have the count file folder in every sample.

I do not what is the origin of question. Hope to get your help. Thank you!

the last row of your lib_list_path.txt (S17765-F11_S36) has switched cellbarcode and CB_ctss_bed column.
the script took your CB_ctss_bed as your cellbarcode lists and found "chr1" as a cellbarcode and complained "chr1" is duplicated.


webpack 5: Invite AdminKit into this century

All deps bumper to latest version
still got 18 vulnerabilities (7 moderate, 11 high)
still got errors in js regarding Chart & flatpickr

Thanks! We'll bump Webpack to v5 in the next minor release, and use some of your code suggestions.

Resolved in #113


Add Strangeworks to partners page

Add "Strangeworks" logo to one-pager and Metriq partners page on website.

Excellent @vprusso @WrathfulSpatula. It also unified the sizes of the logos.


LSC prototype: Split comment on ENTER

Put back the functionality that automatically adds a # at the beginning of next line when you hit ENTER in the middle of a comment
Was previously in EditFilter.cs, in VSConstants.VSStd2KCmdID.RETURN command handler.

Now implemented in PythonReturnKeyCommandHandler


Add a LICENSE.

(Extracting some small things on the way to #20)

This is ok for me, but I'd like to have @jasonrute's opinion too.


No offset commit example

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
I don't see any example of commit a subscribtion offset

We will likely not expose manual offset commits. The preferred use case for Pub/Sub Lite clients is via the Cloud Pub/Sub shim APIs, not through the underlying wire protocols. The shim API abstracts away from needing to care about offset commits.


invalid json response body

Hello I followed the docs and tried both manual & getting started guides but can't figure out the below problem.
I put my API_KEY in env file btw

@codingwithdidem, @joshuawootonn is looking into this and working on much better error messages so please stay tuned! 😄

@codingwithdidem, #121 should fix this issue! Please let me know if you run into it again 😄


Properly reset when pressing Esc in the middle of command

This is a continuation on Adriafarres pull request (#237) since that PR has been idle for the past month and this bug really annoys me. I'll just quote the previous issue description:

Suppose we have the text "|Hello, world", with | being the position of
the cursor. Pressing ddw simply moves the cursor on top of the
comma instead of leaving the text as ", world".
This fixes issue VIM-1421

I've added some more test cases based on the last comment in the other PR.
Please let me know what you think :)

Oh, I like the level of cooperation :)
We have a pre-release freeze, so I'll merge the both PRs right after the release.

Yep, merged. Thank you again for the contribution!


Add NodeBase base class for Node, NodeImage, and NodeSize

Add NodeBase base class for Node, NodeImage, and NodeSize to inherit from in order to share the get_uuid function and common attributes.

Hey, @tildedave actually had a good suggestion on the IRC.
It would be better for NodeBase to be a mixin instead of a base class. Here is an example of a Mixin class we use in tests - https://github.com/apache/libcloud/blob/trunk/test/compute/init.py#L20. Also, better name for this class would be UuidMixin.

Hi, Tomaz. Thank you for your advice. I agree with that suggestion, I'm gonna change and I'll commit again.


Make CPT meta's (title, description e.d) translatable

Currently if you use a multi lingual plugin like Polylang your CPT titles and metadescriptions will be displayed in the default language regarding the language the user is currently browsing your website in.
It would be nice if you can add the translations for those fields in Polylangs String Translation page.
https://polylang.wordpress.com/documentation/setting-up-a-wordpress-multilingual-site-with-polylang/strings-translation/
And see pll_register_string on: https://polylang.wordpress.com/documentation/documentation-for-developers/functions-reference/
And maybe this for WPML users: https://wpml.org/wpml-hook/wpml_register_string/

Het versienummer is nog steeds 0.8.1 dus als 0.8.2 straks uitkomt wordt ie
gewoon geupdate.
2016-05-06 20:39 GMT+02:00 Rob de Kort<EMAIL_ADDRESS>
Currently if you use a multi lingual plugin like Polylang your CPT titles
and metadescriptions will be displayed in the default language regarding
the language the user is currently browsing your website in.
It would be nice if you can add the translations for those fields in
Polylangs String Translation page.
https://polylang.wordpress.com/documentation/setting-up-a-wordpress-multilingual-site-with-polylang/strings-translation/
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/nextbuzz/buzz-seo/issues/58


Issue with HowTo schema

I'm trying to build a HowTo schema like this:
var howToSchema = new HowTo()
{
    Name = "some name",
    Description = "some desc"
};

var step1 = new HowToStep()
{
    Url = new Uri("http://example.com#step1"),
    Name = "Name for step1",
    Text  = "Desc for step1",
    Position = 1
};

var step2 = new HowToStep()
{
    Url = new Uri("http://example.com#step2"),
    Name = "Name for step2",
    Text  = "Desc for step2",
    Position = 2
};

//Here is the issue
//howToSchema.Step property is of type Values<ICreativeWork, IHowToSection, IHowToStep, string>
//I cannot add more than one step, which is the main purpose of the how to element
howToSchema.Step = step1;
howToSchema.Step = step2;

The issue is that I cannot add more than one step.
For reference, google has some examples and the howTo schema has a 'step' array. Here is the link: https://developers.google.com/search/docs/data-types/how-to#standard

Oh nevermind... I just noticed Values can accept an array of IHowToStep items

Glad you were able to work this out yourself @eduardomc ! But yeah, you can use a list or an array and set it to the property - it will automatically convert to the OneOrMany<T> or Values<T, ...> type as appropriate.


Dataset Viewer issue for nyuuzyou/AnimeHeadsv3

Link
https://huggingface.co/datasets/nyuuzyou/AnimeHeadsv3
Description
The dataset viewer is not working for dataset nyuuzyou/AnimeHeadsv3.
Error details:
Error code:   ResponseNotReady

The dataset status has been showing as "ResponseNotReady" for the past six days, and sometimes it briefly changes to "ResponseNotFound". I am not sure if this is a bug or if there is an issue with the dataset itself.

Thanks for reporting. The dataset viewer now shows the root issue:

Maybe @albertvillanova can help on the details of why the dataset is not loading.

Yes, the issue is not with the dataset viewer, but with the dataset: it does not load.
from datasets import load_dataset

ds = load_dataset("nyuuzyou/AnimeHeadsv3")

The reason is that the directory structure within the ZIP archive files is not supported.
images
   ├── 0001_bel1c1i1_jpg.rf.1a4afbb24eedb9665bf5b590b3b7ceed.jpg
   ├── ...
labels
   ├── 0001_bel1c1i1_jpg.rf.1a4afbb24eedb9665bf5b590b3b7ceed.txt
   ├── ...

For image datasets to be supported without a loading script, they need to have a very specific ImageFolder structure. You can get more information here: Create an image dataset.
I have created a discussion in the "Community" tab of the dataset: https://huggingface.co/datasets/nyuuzyou/AnimeHeadsv3/discussions/1
Let's continue the discussion there.
On the other hand, I agree that from the datasets library, we should give a more informative error message.
I am working on this.


Add XML Docs for pubternal types

Part of https://github.com/aspnet/AspNetCore/issues/3656

@davidfowl why didn't we finish making these fully internal?


Sitemap SAX parser mangles sitemap URLs in sitemap index, fixes #169

completely add sitemap URLs from sitemap index if URL contains
XML entities or CDATA
update unit tests to cover this problem

Thanks! Renamed locFound to locClosed to avoid any confusions.


Undo command does not restore deleted cell

Environment data

VS Code version: 1.73.1
Jupyter Extension version v2022.9.1303220346
Python Extension version (available under the Extensions sidebar): 3.9.7
OS (Windows | Mac | Linux distro) and version: Windows 10 homes 64-bit
Jupyter server running: Local

The undo command 'Z' does not work. I am making sure that no cell is active and that the notebook is selected, but nothing happens.

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.85.0. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!


Support private packages

In case your packages require you to login to npm using their cli, can-npm-publish won't see them (instead getting a 404 with node-fetch). I changed the check to use npm view as suggested in #1

@azu Thanks for your feedback, I updated the PR accordingly

@mroswald Thanks!
I've released v1.1.0.


in progress - working on rule 9

If this is in progress, should I wait on the merge?

yes. I made changes in the web interface and didn't know how to create a branch without creating a pull request ...

Hi @dlebauer @emhart once you merge, I have a number of comments on the full draft (which I made on a hard copy) that I'll open as a new PR on the merged version, so as to avoid conflicts...

@naupaka Ok, I'll merge now so you can issue your PR.

good call. Thanks
On Thu, Sep 3, 2015 at 9:56 PM, Edmund Hart<EMAIL_ADDRESS>wrote:

Merged #72
https://github.com/emhart/10-simple-rules-data-storage/pull/72.
—
Reply to this email directly or view it on GitHub
https://github.com/emhart/10-simple-rules-data-storage/pull/72#event-400761442
.


Get the number of references to the given pulsar client instance at a given point of time.

Is your feature request related to a problem? Please describe.
Pulsar Version: 2.7.2
Pulsar Client: Pulsar CPP Client
In Pulsar, we use a single client to create multiple producers/consumers/readers. Is there any method/attribute that can give information on number of producers/readers/consumers connected to the given pulsar client instance at the given point of time?
Say there is a single pulsar client instance. Multiple consumers and readers are created from the given client instance. This client needs to be cleaned up when all the references are closed. In this case, it would be of help, to get information on the number of the consumers/readers/ getPartitionsForTopic calls are active on the given client. Ie, having the number of references for the given client can provide information on whether it is fine to clean up the client instance at the given point of time.
Describe the solution you'd like
A way to get the number of references to the given pulsar client instance at a given point of time.

I'll handle this issue.

I'll add method to ClientImpl:

getNumberOfProducer()
getNumberOfConsumer()
getNumberOfReader()
These method will scan state of all kinds of Producer/Consumer/Reader (with lock) and return number of connected instance.

I have open a PR. There are two questions:

For PartitionedProducer/PartitionedConsumer/MultiTopicConsumer which consist of more than one consumer/producer. They should be counted as one or more reference？
When created reader from client, client add reader's consumer to its consumer list. Should we remove getNumberOfReader()? I haven't implemented this function yet.


Updates, refresh

Validation Status: Warnings
Articles

Article
Status
Details

alm-cordova.md
Warning
Details

alm-cordova.md

[Warning] HTML scripts, styles or image tags are not a valid Markdown syntax. 0 script tags, 1 style tags and 0 image tags were found in the document content.

JobID: dcab2244-b36b-4207-9a8f-61f3416aa8dd

Articles to review
The articles impacted by this PR were staged review at:

http://taco.visualstudio.com/en-us/docs/alm-cordova/

Validation Status: Warnings
Articles

Article
Status
Details

alm-cordova.md
Warning
Details

alm-cordova.md

[Warning] HTML scripts, styles or image tags are not a valid Markdown syntax. 0 script tags, 1 style tags and 0 image tags were found in the document content.

JobID: c273a495-77a6-40cc-8a57-d7ff03cdadf9

Articles to review
The articles impacted by this PR were staged review at:

http://taco.visualstudio.com/en-us/docs/alm-cordova/
http://taco.visualstudio.com/en-us/docs/unit-test-04-chutzpah/


debug a console command

When I am debugging code, sometimes I want to type a ruby expression and have the debugger run on that command so I can see what is happening inside. For this case what I do now is stop the debugger and add the line into the source with a breakpoint in front of it and then rerun my tests. It would be cool if I could have a command like "debug " which would run  and act as if a breakpoint was set on it.

Thanks @cooledge, I'll think about this.

+1 here, it's very useful to use sub-debuggers. Especially when used in the pry-byebug combo. At the moment I'm mostly using pry and only sometimes byebug when debugging commands are important.

Just pushed a fixed to this. Not providing a new command, but just making it work by default.


Fork-test

Description

[ x] I have added a folder(Named it as my GitHub user id)
[x ] I have added a Readme.md file inside the folder
[ ] I understand the GitHub Forking Concept fully

Test comment


migration not working correctly

I created model and migration.and run 'bundle exec rake db:migrate'
and it always migrate sqlite database.
I want to migrate to postgresql, what I don't understand is, it sometime work correctly.

Hi @whitesnow9291 can you please use our template for new issues (see https://github.com/padrino/padrino-framework/issues/new) so that we can have more details about your problem. Giving us the code of your app helps us understanding your problem.
Cheers
Matthias


Add text decorations

Allow a TextView to have special effects on parts of the content only: bold/italic/underline, different color/highlight...

I started working on this. Maybe you can have a look so we agree on the direction to take this in?
You can test the current state by dropping the following in line 244 of TextView and running the lorem example. It just splits the text at "sit" and applies some style. (Yes, this is horrible and only works with the lorem ipsum text)
let delim = "sit";
let mut reappend_delim_i = 0;
let parts = text.split(delim)
    .map(|it| {
        let ret = if reappend_delim_i != 0 {
            format!("{}{}", delim, it)
        } else { it.to_owned() };
        reappend_delim_i += 1;
        ret
    })
    .collect::<Vec<_>>();
let parts = parts.iter()
    .map(|it| it.as_str())
    .collect::<Vec<_>>();
if parts.len() == 3 {
    let part1 = (parts[0], ::style::Style {
        color: None,
        ..Default::default()
    });
    let part2 = (parts[1], ::style::Style {
        color: Some(::theme::ColorStyle::Highlight),
        ..Default::default()
    });
    let part3 = (parts[2], ::style::Style {
        effect: ::theme::Effect::Reverse,
        ..Default::default()
    });
    printer.print_styled((x, 0), &[part1, part2, part3][..]);
} else {
    printer.print((x, 0), text);
}

I'd like, if possible, to separate the parsing part (where marked up text is parsed into a style tree) and the rendering part, to allow the same TextView to be easily filled from HTML, markdown or some other text format.

I was planning to do parsing in its own module where we could implement different parsers for different markup languages.
I don't think we need a tree to store a parsing result. That is only useful if you want to modify the tree (like JavaScript inserting DOM nodes) but that's probably overkill so we can just parse the new markup string instead. The result could be stored as Vec<StyledStr<'a>> as you said in https://github.com/gyscos/Cursive/pull/106#commitcomment-20684775.
My current draft is
/// Handle parsing markup into style.
pub mod markup {
    /// Parses a markup string into `StyledStr`s.
    pub type Parse = for<'a> FnOnce(&'a str) -> Vec<StyledStr<'a>>;

    /// Cursive's own markup language.
    pub fn parse_cursive<'a>(markup: &'a str) -> Vec<StyledStr<'a>> {
        unimplemented!()
    }
}

which would be used in widgets like
// If `None`, disable markup.
markup: Option<Box<markup::Parse>>,

I'm still trying to find how to design it around use-cases like TextView where styled spans are rows are overlapping...

After pondering quite a while I realized that the LinesIterator that computes rows must obviously not operate on the markup string but on the parsed result. That means we need to adapt computation and usage of rows anyway.
I haven't yet thought about this in depth but my first idea is to write a StyledLinesIterator or so with a type Row<'a> = &'a [StyledStr<'a>]. Maybe that StyledLinesIterator could use the already existing LinesIterator so it doesn't need to be completely rewritten.
I'm out earning money until Friday. Looking forward to continue with this.

Let's start from the end.

The Printer takes a &[StyledStr<'a>] per row.

We want to store the rows (to avoir re-computing them) but storing these StyledStr<'a> inside TextView might be tricky (self-borrowing types are never easy to deal with).
So instead, we'll store a list of struct StyledSpan { style: Style, start: usize, end: usize, width: usize }. It's then trivial to build the list of StyledStr<'a> from that when drawing.

Our StyledLinesIterator should then output StyledSpans. It will need to:

Accumulate StyledSpans as long as they fit on one row
Split the last span following the current rules
Start the next row with the leftovers from the split span.

The input to the StyledLinesIterator would then be a styled input, made of:

A list of StyledSpans
The input string the spans are indexing into

(Keeping the string separate and using indices in there is just a trick to avoid issues with storing references in the string; if there is another option - I haven't checked Kimundi/owning-ref-rs yet - that could change.)

So all in all, a TextView would have to store:

The main input string
The result of the style parsing: a list of StyledSpan
Pre-computed rows: a list of lists of StyledSpan

And when asked to draw itself, it would map a list of StyledSpan to a list of StyledStr<'a> using the input string, and give that to the printer...

That sounds good! I'll start working on it when I have some time.
I think our parse() signature from dermetfan@bf43333 (comment) is unable to parse markup with escape sequences.
Consider this HTML: <i>1 < 2</i>
Using this signature, our parser could only return:
[(
  "1 < 2",
  Style {
    color: None,
    effect: Effect::Italic
  }
)]

Since parse() just has a string slice it cannot transform this into:
[(
  "1 < 2",
   ...
)]

A Cow solves the problem: parse<'a>(markup: &'a mut Cow<str>) -> Vec<StyledStr<'a>>
So to update the example from my previous comment:
/// Handle parsing markup into style.
pub mod markup {
    use super::*;
    use std::borrow::Cow;

    /// Parses a markup string into `StyledStr`s.
    pub type Parse = for<'a> FnOnce(&'a mut Cow<str>) -> Vec<StyledStr<'a>>;

    /// Cursive's own markup language.
    pub fn parse_cursive<'a>(markup: &'a mut Cow<str>) -> Vec<StyledStr<'a>> {
        unimplemented!()
    }
}

Aah damn, I forgot about that. Cow seems perfectly indicated in this case indeed.

Oh wait - should the input itself be Cow, or just the content of StyledStr<'a>?

I meant the input into parse() but yes type StyledStr<'a> = (Cow<'a, str>, Style) is probably better. That would allow to easily parse the tags producing a [StyledStr] and hand that off to a function that replaces the escape sequences. We get an owned version for free :)

Looks strange with termion, crossterm backends:

Weird indeed!
Does this also happen with the other examples? It might be caused by a font misalignment... :S

Weird indeed!
Does this also happen with the other examples? It might be caused by a font misalignment... :S

No, other examples looks normal.


Error with Permissions-Policy header: Unrecognized feature: 'ch-ua-form-factor'.

Error with Permissions-Policy header: Unrecognized feature: 'ch-ua-form-factor'.

Bonjour,
Origine du problème ? Comment le régler ?
Merci

This appears to be spam, closing.


Separate pairing from connection for Canon EOS

Thanks to @wolcano for delving into the Canon EOS more.
It looks like pairing and connecting for Canon can and should be separated, especially due to an 'acceptance' bit where the camera needs an 'OK' to complete the pair.
For Canon EOS, the pair acceptance result comes via a BLE notification.
Which seems to be:

service UUID 00010000-0000-1000-0000-d8492fffa821
characteristic 00010006-0000-1000-0000-d8492fffa821
indicator value 0x02 -> accept
indicator value 0x03 -> reject

This is consistent with the efforts from https://github.com/robot9706/CanonBLEIntervalometer

Turns out there's no need to separate the pair from connect for now.
The camera sends an immediate 'accept' on the paired re-connect.


zwave health check endpoint not indicating a zwave failure

Checklist

[X]  I am not using Home Assistant. Or: a developer has told me to come here.
[X] I have checked the troubleshooting section and my problem is not described there.
[X] I have read the changelog and my problem is not mentioned there.

Deploy method
Docker
Z-Wave JS UI version
8.10.1
ZwaveJS version
10.11.1
Describe the bug
When zwavejsui loses connection to a TCP based zwave controller, /health/zwave does not indicate an issue. Not sure how locally connected usb controllers are affected.
As stated in Health check endpoints the http return code should be 200 if zwave client is connected.
wget command run from container
wget --server-response --spider --quiet --no-check-certificate "http://localhost:8
091/health/zwave" --header "Accept: text/plain" 2>&1 | awk 'NR==1{print $2}'

response
200

PS the command listed in https://zwave-js.github.io/zwave-js-ui/#/guide/healthcheck?id=health-check-endpoint does not run on the container as curl does not exist.
Recommend using the command above instead.
Edited: removed browser response as the healthcheck endpoint is not meant to be viewed in a browser.
To Reproduce
With the zwavejsui container running, reboot the tcp zwave controller. Then operate a device once the tcp controller reboots.
Expected behavior
Expect the /health/zwave page to indicate zwave is down as the connection to the controller is down.
Additional context
Logs when connected to tcp controller

Logs after tcp controller has been rebooted

@mwolter805 As described in docs the url must specify the header "Accept: text/plain"  in order to work, if not present (like when you open a web page) it is considered as a html request so it redirects it to frontend

@robertsLando Thank you for the response, however, please see the command and output above the picture.

@mwolter805 as I sad that healthcheck endpoint is not meant to be served on frontend, it's just an api that can be called with curl or what you prefer

@robertsLando thank you for the reply, yes I understand what you are saying, the healthcheck endpoint is not meant to be viewed from the browser. I have updated the issue and removed the browser issue.
However, I believe the issue with the healthcheck endpoint returning http return code 200 when zwave is down has not been addressed. Please see the command and output above. When zwave was down the healtcheck endpoint returned http return code 200 when none of the zwave devices (controller or switch) were operational.

Ok got it. I think this is a known issue on zwave-js driver side not detecting tcp errors: https://github.com/zwave-js/node-zwave-js/issues/4660
cc @AlCalzone

@robertsLando @AlCalzone Thanks, Zooz released an inexpensive 800 series controller hat ($25) for the Raspberry Pi that works well, except for this. As far as I can tell the Pi and controller are working well and this is the last major thing. Having an option for a controller over wifi or ethernet is very desirable and planning to do writeup on the forums for this once it's reliable.

Correct, I'll need to figure out what broke in the detection of TCP disconnects.

@mwolter805 Can you follow updates on that issue please? Juswt subscribe to it, I close this one as it's not a zui issue and it's a dup of that

@robertsLando Sure thing, no problem. Thanks for your help.


Daisy cannot recover from failed GetSerialPortOutput failures.

It looks like if there's multiple GetSerialPortOutput failures, Daisy will halt the process.
https://github.com/GoogleCloudPlatform/compute-image-tools/blob/dc64ab897d7331b29217d4df6b4cd9cb9d304d1e/daisy/step_wait_for_instances_signal.go#L144

Hi Jeremy, this is correct. Are you affected by this? If so, thoughts on alternatives?

Interim fix:

Use exponential backoff with randomized jitter. It looks like something sorta like that is going on: https://github.com/GoogleCloudPlatform/compute-image-tools/blob/9ae272235f86b6a3dbd657b259ca308126b3288a/daisy/compute/compute.go#L210 but I'm not sure I'd roll my own retry loops there's plenty of golang libraries for this.
Make the retry logic not fail after 3 times. Make it time bound like say if I keep getting failures for 5 minutes then give up.

Permanent Fix:
Read from

Stackdriver logs as the primary source. If there's no access fallback to GetSerialPortOutput with a less aggressive polling internal. Once per minute instead of once every 10 seconds.

These all sound like reasonable approaches. If you're interested in writing a PR, we could chat about specifics, and I think @adjackura  would have some thoughts too.
Before that, I'm curious: How often do your builds fail due to this?


Update redis to 0.13

This doesn't update the lockfile because I'm at work and don't have proper access to git! I did test that everything still compiles, though.

Thanks, updated lockfile and released as 0.12.


Capital eszett not rendered correctly

Steps to Reproduce

create flutter project
add Text Widget
put an capital eszett (ẞ) in it

Logs
[✓] Flutter (Channel stable, v1.7.8+hotfix.4, on Linux, locale en_US.UTF-8)
    • Flutter version 1.7.8+hotfix.4 at /home/gregor/development/flutter
    • Framework revision 20e59316b8 (4 weeks ago), 2019-07-18 20:04:33 -0700
    • Engine revision fee001c93f
    • Dart version 2.4.0

[!] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /home/gregor/Android/Sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /snap/android-studio/77/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b16-5323222)

[✓] Android Studio (version 3.4)
    • Android Studio at /snap/android-studio/77/android-studio
    • Flutter plugin version 38.2.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b16-5323222)

[✓] Connected device (1 available)
    • Android SDK built for x86 • emulator-5554 • android-x86 • Android 9 (API 28) (emulator)

Analyzing schoolsen-mobile...
No issues found! (ran in 21.3s)

I tried rendering this character on a few devices and it looked as expected.
Do you have a code example that reproduces this?  Does it only happen on a specific device?

Sorry it was my fault. I used a font that doesn't support this character.


Commenting attributes in JSX

I couldn't find a way to change the behavior of auto-commenting (cmd + /) when the cursor is inside a JSX element's attributes. Webpack is throwing a build error when I comment out an attribute (for example, if I want to test an element without the className attribute) because it puts {*  *} around the line, but apparently this is an "unexpected token".
Comments using regular // or /* */ are valid when inside the angled brackets of an element declaration, e.g.:
<MyComponent
  // className="my-class"
  myProp={propValue}
/>

Is there a way to enable this type of behavior selectively, or would it be difficult to implement?

I do this automatically. My guess is you aren't using my grammar. Should show Babel ES JavaScript on the status bar.

From: Phil<EMAIL_ADDRESS>Sent: 10/06/2016 16:30
To<EMAIL_ADDRESS>Subject: [gandm/language-babel] Commenting attributes in JSX  (#206)
I couldn't find a way to change the behavior of auto-commenting (cmd + /) when the cursor is inside a JSX element's attributes. Webpack is throwing a build error when I comment out an attribute (for example, if I want to test an element without the className attribute) because it puts {*  *} around the line, but apparently this is an "unexpected token".
Comments using regular // or /* */ are valid when inside the angled brackets of an element declaration, e.g.:
<MyComponent
  // className="my-class"
  myProp={propValue}
/>

Is there a way to enable this type of behavior selectively, or would it be difficult to implement?

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/gandm/language-babel/issues/206

Sorry just re-read this and i'll check.

Thanks! Appreciate it.

I just pushed 2.28.0 that has improved the commenting of JSX. Hopefully, it will use the most appropriate comment type depending on the start line being commented. e.g. attributes will use //

Seems to work great. Thanks again.


Have a version which is all static functions

Due to the way our deployment works, I cannot use this because of the way the library is designed.
It would be great if the FixidityLib was purely functional, such that all values are passed in.  I.e.
library FixidityLib {
    function (...) internal pure view {
    }
}

That way it's very flexible in how it can be used.  Whether as an external library, contract, or otherwise.
In the meantime I've simply copied the code over.

Hello,
thanks for the report. Can you explain a bit further about "the way our deployment works"?
Cementdao is using fixidity to its project and depending on the deployment pipeline it may work for some and not for others, and obviously, that is not the objective.
And I apologize, but I didn't understand the question It would be great if the FixidityLib was purely functional. It is purely functional and we are only passing values to functions.

static functions is a term not commonly used with Solidity, so it's kind of a misnomer here.
What I meant was if all the functions were internal pure view so that if the user wants to they can in-line the functions into a contract, rather than have to deploy the whole contract.
Because the functions are all public pure view the library must be deployed first and then linked to.
It would be nice to have the option to simply inline the functions.
As for the deployment, looking a little further into ZeppelinOS revealed that it can handle library linking and deployment automatically.  It wasn't evident to me, but it works!


[DOC-3501] Add SMP advanced configurations section, new external db overview topic

Add SMP Advanced configurations section, new external db overview topic
For reviewers, a preview is available:
If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[x] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app


jqLite text() function has different behavior then jquery text() function

angular.element('

').text(undefined)

is returning empty string ("")
$('

').text(undefined)

is returning DOM object (Object[div])
Tested version: 1.3.15

This actually affects way more APIs than just .text(). We'd have to review them all & update.

Since we're reaching feature freeze, this is a won't fix at this point. Sorry!


On resolveKeys failure, refetch JWK Set cached by the RP in case stale.

AuthenticationResponse#resolveKeys may fail with error 'Cannot resolve signing key for ID Token' if the OP JWK Set cached by the RP is stale due to key rotation by the OP. Google changes its public keys around once per day. (https://developers.google.com/identity/protocols/OpenIDConnect)

Thanks, could you please merge in current master, so that it brings in the fix for my Travis messup? :-)

@dmitrizagidulin /cc @kjetilk @kidehen: Please can you review this PR or give an indication of when it might be reviewed. Thanks.

@cblakeley Can you rebase?

@RubenVerborgh @kjetilk: I opted to merge upstream/master into my PR branch, rather than risk a git push --force following a rebase. OK now?

@cblakeley yeah, seems OK! @dmitrizagidulin perhaps something we could look into on tomorrow's scheduled time?

I opted to merge upstream/master into my PR branch, rather than risk a git push --force following a rebase. OK now?

A rebase (with a force push) is almost always preferable to merging upstream master - it's a lot cleaner, less confusing, etc. Don't be afraid of rebasing! :)
@kjetilk

@dmitrizagidulin perhaps something we could look into on tomorrow's scheduled time?

Sure!


IConfiguration.GetChildren() time grows exponentially based on number of keys to read

This issue has been moved from a ticket on Developer Community.

In the process of taking legacy applications and getting them to a state where they will run in a cloud environment, we ended up having a configuration file with ~25,000 unique keys.  (Multi-tenant application with thousands of tenants each having different configuration... it gets a bit big).  We ran into an issue where reading through all of the configuration keys was taking a long time.  While investigating the issue I narrowed it down to IConfiguration.GetChildren().
After doing some testing on key volume to time, I can report the following:
Reading ~1000 keys took about 0.1s
Reading ~2000 keys took about 0.23s
Reading ~4000 keys took about 1s
Reading ~25000 keys took about 40s
We are already changing the way we store/read configuration, but I felt it worth reporting since the key to time ratio is not linear.

Original Comments
Roi Chen [MSFT] on 10/15/2019, 08:25 PM:
Please help to reassign if the current area path is not proper. Thanks.
Visual Studio Feedback System on 10/21/2019, 04:05 AM:
This issue is currently being investigated. Our team will get back to you if either more information is needed, a workaround is available, or the issue is resolved.

Original Solutions
(no solutions)

It seems that this issue is partially related to the linear representation of data in the ConfigurationProvider. I think that tree-based implementation of IConfigurationProvider should be better. I could try to make a tree-based implementation to compare it with the current implementation. After that I could create an issue with details of the idea.

@AMalininHere were you able to reproduce the issue?
My basic test is seeing close to linear (probably n log n) performance of GetChildren() -- 40s for 25,000 keys is a really long time given the context.  There has to be something else going on beyond what was reported.
--- a/src/Configuration/Config/test/ConfigurationTest.cs
+++ b/src/Configuration/Config/test/ConfigurationTest.cs
@@ -332,6 +332,29 @@ public IConfigurationProvider Build(IConfigurationBuilder builder)
             }
         }

+        [Fact]
+        public void GetChildrenPerformanceTest()
+        {
+            var dict = new Dictionary<string, string>();
+            var random = new Random();
+
+            for (int x = 0; x < 25000; ++x)
+            {
+                dict[$"abc.123." + random.Next().ToString()] = "foo";
+            }
+
+            var memConfig = new TestMemorySourceProvider(dict);
+
+            var config = new ConfigurationBuilder()
+                .Add(memConfig)
+                .Build();
+
+
+            _ = config.GetChildren()
+                .ToArray();
+
+        }
+
         [Fact]
         public void SettingValueUpdatesAllConfigurationProviders()
         {


Slider default

Hello natan,
Thanks for sharing those, pretty (pretty) deep and very interesting to learn.
I added the ability for sliders to have a default value that can be recalled via middle-clic. As a consequence, sliders can now only be dragged via left-drag. I also thought they could be fine-tuned via right-drag, but that'd be an other PR.
I also took the liberty to relink the nodes in the Spreading demo, they were still referencing the ones with the old names (UIxxx).
What would you think of that ?
Cheers.

Ah, did not see that CTRL+Drag already fine-tunes the value, cool!


Add and expose metrics in ObjectStore Trait

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to  the what)
The current ObjectStore Trait and implementations do not collect and expose any metrics.
Add some metrics like read/write/delete ops counts, scan bytes,  elapse times etc and expose them will be very useful
in the production environment.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

One way to do this might be to implement a wrapper around an inner Arc<dyn ObjectStore> interface that had metrics.
Much like https://github.com/apache/arrow-rs/blob/master/object_store/src/limit.rs or https://github.com/apache/arrow-rs/blob/master/object_store/src/throttle.rs
BTW I plan to release a new object_store release (from arrow-rs) mid next week


Edit Diagram and Manual Testing in DG

Manual Testing section in DG is now populated. This enhancement:
Resolves #296
Resolves #336
Resolves #337
Resolves #314
Resolves #333 - Use case phrasing now clearer
Resolves #332 - Unused section is deleted
Resolves #331 - Unused diagram is deleted
Model Diagram is edited. This enhancement:
Resolves #329
Resolves #328
Resolves #330

Looks good to me!


Updater

Add an updater that will use wget or curl to pull the page source for the "Latest" tag. From there try and do a search to see if the version is greater than the currently installed. If it is, pull the download link and update itself and the files required.

Is included in version 0.3.0 on testing. Will be pushed to master once "fully" completed.


line 1: cannot unmarshal !!str <%!(MISSING) into director.Manifest

Hi,
I am try to deploy sas on cloudfoundry and I am facing the below error.
ERROR:
#bosh -e bosh-2 -d sas-viya deploy SAS-Viya-manifest.yml
Using environment '<IP_ADDRESS>' as client 'admin'
Using deployment 'sas-viya'
Parsing manifest:
Unmarshalling manifest:
yaml: unmarshal errors:
line 1: cannot unmarshal !!str <%!(MISSING) into director.Manifest
Exit code 1
Can anyone help with this. Your help will be much appreciated.
Below are the details of bosh CLI and director.
Bosh CLI Details :
#bosh --version
version 5.1.2-14b04789-2018-08-17T15:15:17Z
Bosh Director details :
#bosh -e bosh-2 environment
Using environment '<IP_ADDRESS>' as client 'admin'
Name bosh-bbl-env-winnipeg-2018-09-27t21-06z
UUID 7404a17f-d938-432c-ae33-00575bcffcb2
Version 267.5.0 (00000000)
CPI aws_cpi
Features compiled_package_cache: disabled
config_server: enabled
dns: disabled
snapshots: disabled
User admin
Succeeded
Regards,
Srikanth

It looks like you're trying to use a very old manifest which is using ERB-style interpolation. ERB interpolation is no longer supported with recent versions of the CLI. I'd recommend you take a look at this interpolation doc to learn more and, as a reminder, you no longer need to specify director_uuid in your manifest.

Thanks for the information.
I have deployed the bosh director using the bosh bootloader and I am able to connect using bosh CLIV2 and deployed CF, but now I cannot change the SAS Manifest file so I have to go with it.
I have now installed the bosh CLIV1
[root@bosh bbl]# boshv1 --version
BOSH 1.3262.26.0
but when I try to connect to bosh director using CLIV1 commands, I am facing below error.
[root@bosh bbl]# boshv1 target https://<IP_ADDRESS>:25555
Target set to 'bosh-bbl-env-winnipeg-2018-09-27t21-06z'
/usr/local/share/gems/gems/cf-uaa-lib-3.2.5/lib/uaa/http.rb:173:in rescue in net_http_request': error: Connection timed out - connect(2) (CF::UAA::BadTarget)         from /usr/local/share/gems/gems/cf-uaa-lib-3.2.5/lib/uaa/http.rb:159:in net_http_request'
from /usr/local/share/gems/gems/cf-uaa-lib-3.2.5/lib/uaa/http.rb:147:in request'         from /usr/local/share/gems/gems/cf-uaa-lib-3.2.5/lib/uaa/token_issuer.rb:77:in request_token'
from /usr/local/share/gems/gems/cf-uaa-lib-3.2.5/lib/uaa/token_issuer.rb:264:in client_credentials_grant'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/client_token_issuer.rb:18:in access_info'
from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/client.rb:34:in block in access_info'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/client.rb:50:in with_save'
from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/client.rb:34:in access_info'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/token_provider.rb:47:in client_access_info'
from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/token_provider.rb:33:in get_access_info'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/uaa/token_provider.rb:16:in token'
from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/client/credentials.rb:14:in authorization_header'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/base_command.rb:72:in logged_in?'
from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/commands/misc.rb:119:in set_target'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/command_handler.rb:57:in run'
from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/lib/cli/runner.rb:59:in run'         from /usr/local/share/gems/gems/bosh_cli-1.3262.26.0/bin/bosh:19:in <top (required)>'
from /usr/local/bin/boshv1:23:in load'         from /usr/local/bin/boshv1:23:in '
Can you help me to connect to bosh director using the CLI1 commands.

Unfortunately, CLIv1 is no longer supported (CLIv2+ has been available and stable for about a year and a half). I'd recommend investing some time to migrate. There should be minimal changes needed, and some command-specific migration notes can be found here.
In your specific case, the environment bbl creates causes you to rely on bosh's v2 capabilities for tunneling traffic to your VMs. This is not something that the v1 CLI supports. Once you have upgraded your tools and manifests, you should be able to get further. If, after upgrading, you experience new errors, please feel free to open a new issue.

Hi,
Now I am using cliv2 supported Manifest file but I am facing the below error. I have attached manifest and bosh release output command file.
Error: Unable to render instance groups for deployment. Errors are:

Unable to render jobs for instance group 'cas_controller'. Errors are:

Unable to render templates for job 'cas_controller'. Errors are:

Error filling in template 'cas_grid_vars' (line 14: undefined method `ip' for nil:NilClass)
Error filling in template 'properties.sh.erb' (line 29: undefined method `ip' for nil:NilClass)

Unable to render jobs for instance group 'microservices'. Errors are:

Unable to render templates for job 'sas-microservices-deploy'. Errors are:

Failed to find variable '/bosh-bbl-env-winnipeg-2018-09-27t21-06z/sas-vdmml-ubuntu/sas' from config server: HTTP Code '404', Error: 'The request could not be completed because the credential does not exist or you do not have sufficient authorization.'

Thanks in advance.
manifest.txt
releases.txt


Assigning words to cells in TextObjectTable is not always reliable

In case where an actual cell has a tab or a large space between words, these words may be assigned to multiple columns, which creates an extra column just for these words and breaks the table.

TextObjectTable.CSM is one way to handle this, but a large value can lead to columns with small distance between them merging together.

Possible to perform column allocation multiple times with different CSM values and then select one automatically somehow.

After words have been allocated to columns, we can decide to reduce columns with too few rows that contain something. This may not be a good option as it may reduce actual columns.

Possible to allow supplying a regexp per each column for additional validation, which could allow to reduce fake columns.

Also need to keep in mind that columns can be aligned left, center and right.


Add support for B2CUserFlow CRUD operations

This commit adds the following

B2CUserFlowClient for interacting with /identity/b2cuserflow endpoint
B2CUserFlow model for the request/response payload
Some tests for the CRUD operations

Hello @manicminer :wave:  ! Can you please review this PR?


Missing deps on C65

Undefined symbols for architecture x86_64:
  "PermissionRequest::PermissionRequest()", referenced from:
      (anonymous namespace)::AttestationPermissionRequest::AttestationPermissionRequest(GURL const&, base::OnceCallback<void (bool)>) in cryptotoken_private_api.o
  "TrustedSourcesManager::Create()", referenced from:
      DownloadPrefs::IsFromTrustedSource(content::DownloadItem const&) in download_prefs.o
  "PermissionRequestManager::AddRequest(PermissionRequest*)", referenced from:
      extensions::api::CryptotokenPrivateCanAppIdGetAttestationFunction::Run() in cryptotoken_private_api.o
  "content::WebContentsUserData<PermissionRequestManager>::kLocatorKey", referenced from:
      content::WebContentsUserData<PermissionRequestManager>::UserDataKey() in cryptotoken_private_api.o
  "PermissionRequest::GetGestureType() const", referenced from:
      vtable for (anonymous namespace)::AttestationPermissionRequest in cryptotoken_private_api.o
  "PermissionRequest::GetContentSettingsType() const", referenced from:
      vtable for (anonymous namespace)::AttestationPermissionRequest in cryptotoken_private_api.o

this is when building debug build, to be specific is when we turn off optimization


How to integrate C# into the project?

As @Jerry-Gren says, sometimes the power of C# needs to be unleashed since it is the native language of our target. But how?

Currently, there isn't a perfect solution to this problem, so perhaps this issue should be temporarily closed.


Kestrel in LAN + IIS in DMZ via reusable reverse/duplex connections

Is your feature request related to a problem? Please describe.
I am trying to expose one or more applications running in the LAN on Kestrel, via an IIS running in the DMZ to the internet without opening any manual firewall holes from the DMZ machine to the LAN machine.
Currently you need to operate Kestrel and IIS on the same machine.
Describe the solution you'd like
Basic approach
It would be great if you could tell the Kestrel IIS integration on which machine the IIS is running. Then Kestrel connects from the LAN into the DMZ (which is allowed from security perspective) and hooks up with the IIS. The IIS can then use this TCP connection to tunnel any incoming HTTP/WSS calls to Kestrel in the backend.
Advanced approach
Kestrel opens 1 control and multiple data connections as connection pool. Via the control connection some keep-alive mechanism ensures that everything is working and via control commands additional data connections can be opened or closed if depending on the needs.
Describe alternatives you've considered
Currently one of our application has a manually developed API gateway that is tunneled via a duplex WCF channel. The same WCF contract we have in the backend, is exposed on a service in the DMZ and calls are simply tunneled through. The web application, also running in the DMZ can then call against the "proxy" to reach the LAN service.
For another new ASP.net core application, we have created also such a duplex WCF channel but with a special pipeline: In LAN service we have a custom IServer with this WCF duplex connection to the DMZ. On the DMZ service we have a minimal OWIN host, that simply forwards the OWIN context (dictionary) via WCF call to the LAN. This DMZ service is very thin and simply tunnels calls on HTTP/OWIN context level. So when a HTTP call comes to our DMZ service, we unwrap the OWIN dictionary, make a call to the backend via the duplex channel using this dictionary (and special streams) where it goes through the whole ASP.net pipeline. The response is then unwrapped again on the DMZ service and the data is forwarded to the calling client.
Additional context
Overall I would like to simplify our application stack to have a classical ASP.net core web application development. No special cross service communication, no API gateway, no special deployment. Via this proposed small reverseproxy module on top, it can bring this LAN/DMZ separation feature for exposing a local service via a remote IIS.
In the existing out-of-process hosting model, the ASP.net core module already tunnels HTTP calls to Kestrel running in another process. I guess this is done by simply opening a connection to http://localhost:{port} from IIS to Kestrel and then tunnel the data. Simply spoken there would just be a need for a special connection pool that is used for communication between IIS and Kestrel. This connection pool that can be built-up in 2 ways: The current way that IIS opens connections directly as by today. The new way proposed in this issue would tell Kestrel to pre-open a configured number of connections that are then re-used. Dynamic scaling of the pool could then be an advanced feature on top.
Many people might be confused on this topic and ask why this is really needed: There are many security critical environments where you cannot open ports between LAN and DMZ. There is also some IT maintenance effort to setup and maintain such even if it would be allowed to open individual ports. The only solution for this today is to invest more development effort on application level to have this HTTP/WS tunneling in a semantic fashion via an API gateway.
Please note that this feature also might bring other use cases like load balancing and scalability: If there are multiple Kestrel applications running in the LAN (e.g. via dynamically scaled docker instances), they could all register them on the IIS and the IIS does the load balancing between the available services.

We have no plans to add this kind of connection model to Kestrel directly, it doesn't really fit our model. If you need something like this, I expect there is proxy software out there than can do this kind of reverse tunneling (and Windows has a feature for it) but it's well beyond our scope.


ref: restore eval loop

Hello @williamFalcon! Thanks for updating this PR.

In the file pytorch_lightning/callbacks/base.py:

Line 149:76: W292 no newline at end of file


Add another maintainer for extract-text-webpack-plugin

I think there are a bunch of issue that need to be addressed.
There are also some pull requests 6+ months old that probably should be merged.
@sokra if you do not have the bandwidth to take care do this, do you mind, do add another maintainer?

@sokra if you do not have the bandwidth to take care do this, do you mind, do add another maintainer?

Do you want to volunteer?

@ChrisCinelli Let me know if you want to pitch in still. We have more active effort going on and we can support more developers.


[METAL] Fix flaky memory issue due to racing

This PR aims to fix a relatively rare flaky race issuse when we are attempting to purge memory and the buffer is still in use in command buffer

The CI issue seems to be irrelevant. So will go with merge.


Alternative Cluster Architecture

If I have understood how deepstream works in a cluster it does the following:
(order is not entirely correct)
Node 1 receives a record update
Node 1 writes to cache
Node 1 writes to storage
Node 1 sends record update message to ALL other nodes
Node 1 notifies interested clients
All other nodes receives update and notifies interested clients

The last part where the message is sent to all other nodes is a bit of a bottleneck. It also limits the scalability of the system as it is not quite linear in terms of amount of data being sent between nodes.
I wonder if it would be a good idea to instead:
A. Send only record meta (id, version) and not record data to other nodes, which have to fetch data from cache if they have interested clients. This would require a new type of record message in the protocol, e.g. INVALIDATE.
B. Have all nodes listen to a change stream from the cache connector and only read in the record if there are any clients interested.
C. Partition records across nodes in the cluster by hashing their id.

I think A is easily achievable and could provide a nice performance boost.

@yasserf: I've implemented option D in our fork. Basically what we do is:
constructor() {
  ...
  this._subscriptionRegistry.setLocalSubscriptionListener({
    onLocalSubscriptionMade: name => this._message.subscribe(buildTopic(C.ACTIONS.UPDATE, name), this._broadcast),
    onLocalSubscriptionRemoved: name => this._message.unsubscribe(buildTopic(C.ACTIONS.UPDATE, name), this._broadcast)
  })
  ...
}

_broadcast (nextRecord, sender) {
  ...

  this._subscriptionRegistry.sendToSubscribers(
    nextRecord[0],
    messageBuilder.getMsg(C.TOPIC.RECORD, C.ACTIONS.UPDATE, nextRecord),
    sender
  )

  if (sender) {
    this._message.publish(buildTopic(C.ACTIONS.UPDATE, nextRecord[0]), nextRecord)
  }
}

function buildTopic (action, name) {
  return `${C.TOPIC.RECORD}.${action}.${name}`
}

So basically the message connector is used to only send updates to nodes that are interested instead of sending it to all nodes.


Agent not starting after upgrading to Docker 1.10.2

Rancher Version: v0.51.0
Docker Version: 1.10.2
OS: Ubuntu 14.04.4
Steps to Reproduce: Upgrade from Docker 1.6.2 to Docker 1.10.2
Results: Agent does not start (see error below)
Expected: Agent starts up properly.
After upgrading Docker to 1.10.2 (in preparation so I can upgrade to rancher > v0.59.x) I am finding that the agent will not start up:
ashinn@cloud-transfer4:~$ d logs --since 30m 204
2016-02-29 17:00:13,463 ERROR agent [140466666328304] [event.py:111] 966d2026-3d66-46d4-b5da-3b9a8b27cccb : Unknown error
Traceback (most recent call last):
  File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 94, in _worker_main
    resp = agent.execute(req)
  File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
    return self._router.route(req)
  File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
    resp = handler.execute(req)
  File "/var/lib/cattle/pyagent/cattle/plugins/core/event_handlers.py", line 32, in execute
    type.on_ping(event, resp)
  File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 125, in on_ping
    self._add_instances(ping, pong)
  File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 137, in _add_instances
    running, nonrunning = self._get_all_containers_by_state()
  File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 170, in _get_all_containers_by_state
    for c in client.containers(all=True):
  File "/var/lib/cattle/pyagent/dist/docker/api/container.py", line 67, in containers
    res = self._result(self._get(u, params=params), True)
  File "/var/lib/cattle/pyagent/dist/docker/client.py", line 146, in _result
    self._raise_for_status(response)
  File "/var/lib/cattle/pyagent/dist/docker/client.py", line 142, in _raise_for_status
    raise errors.APIError(e, response, explanation=explanation)
APIError: 500 Server Error: Internal Server Error ("engine is shutdown")
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
...

It is stuck in a constant restart:
ashinn@cloud-transfer4:~$ d ps
CONTAINER ID        IMAGE                  COMMAND             CREATED             STATUS                              PORTS               NAMES
2046905cf9c9        rancher/agent:v0.8.2   "/run.sh run"       3 months ago        Restarting (1) About a minute ago                       rancher-agent

I tried adding the agent again (without removing anything) and the initial start fails with:
...
INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx
INFO: ENV: CATTLE_SYSTEMD=false
INFO: ENV: CATTLE_URL=https://rancher.mydomain.com/v1
INFO: ENV: DETECTED_CATTLE_AGENT_IP=<IP_ADDRESS>
INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v0.8.2
INFO: Deleting container rancher-agent
invalid value "." for flag -v: . is not an absolute path
See 'docker run --help'.

I'll next try rebooting the server and clearing /var/lib/rancher. But any idea why this is happening?

I see that issue some time ago and after some search i run this command.
sudo rm -fR /var/lib/docker/network/*

I ultimately had to rebuild this agent after the server wouldn't reboot due to a disk issue. But I'll try this after I encounter it again and report back. Thanks for the suggestion!

I'm reopening as this is happening again on another test host and removing /var/lib/docker/network/* didn't help.

What I ultimately have to end up doing for these servers is:

Kill all the rancher containers (even the stopped ones)
Remove the images
Run the agent run command again for the environment

I left the /var/lib/rancher/state and everything seems to come back up after this. I can't really explain why removing the images is necessary. But just killing the containers and running the agent did not work. I'm not sure if this is related to Rancher or Docker but since I have a workaround now I'm OK with this being closed (at least here in case someone else runs into the same thing).
I'm basically on v0.56 still and trying to get everything on Docker 1.9+ before going to v0.59+. It seems like all my Docker 1.6 hosts have this issue going to Docker 1.10.2.

Docker 1.9.1+ is our new requirement for Rancher (as of v0.59.0), so moving them all to at least Docker 1.9.1+ would be suggested before upgrading.
https://github.com/rancher/rancher/releases/tag/v0.59.0
Going to close for now.

I encountered this issue (invalid value "." for flag -v: . is not an absolute path) on Rancher agent launch after an upgrade from Docker 1.6 to 1.12.1 on Ubuntu 14.04. Killing all containers and removing all images did allow me to successfully run the agent.


Toaster component issue

I am trying to use toast component in a project.
But i am getting this error
Before this error there was another error that useCallback is not exported from React.
I configured the component.json file to fix issue but not getting the following error:
This is only with the toast component for now. Other components such as Button are working fine.

Stack Trace:
./node_modules/<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>./src/src/components/ui/toast.tsx
./src/src/components/ui/toaster.tsx

Project :  https://github.com/Shashank03200/next-reddit

Hey @Shashank03200,
I just cloned your project locally, ran yarn install and then yarn dev and everything seemed to run okay for me. I did have to change the location of the import of the useToast hook in the Toaster component from import { useToast } from "@/src/components/ui/use-toast" to import { useToast } from "@/components/ui/use-toast" to match the import in the UserAuthForm component.
Maybe try doing a fresh install of your project dependencies and seeing if that works. Probably also worth sticking to a consistent component folder structure (you have some in /src/components/ui and some in /src/src/components/ui).

This issue has been automatically closed because it received no activity for a while. If you think it was closed by accident, please leave a comment. Thank you.