Using Headless Browsers

It seems like a good idea to integrate headless browsers using a Rust crate such as thirtyfour to allow for fast, performant, and easy access to existing plugins that rely on websites without needing API keys for setting up tools such as google, which can be complicated for some users. It could also make it much easier to implement new tools and APIs.

I propose this combination:

https://github.com/rust-headless-chrome/rust-headless-chrome
get webpage, run javascript
https://github.com/kumabook/readability/tree/master
use it to get useful text from the webpage

My reason:

rust-headless-chrome is lighter, and more future-proof, it is being actively developed
need some lib to clean up webpages, I used browser reader mode plugins based on readability type of lib, they served me well as an end-user historically
I also used nodejs version of readability previously for my pet project, it worked fine

I'll spend some time gluing together something from both lib and show you the result in a day or two

@Cormanz I tinkered for some time, encountered some difficulties:

nodejs lib @mozilla/readability is mature and performs well, however their rust equivalents are not
the straight port https://crates.io/crates/readability is dated and the project is lack of activity
another more active and long standing project https://crates.io/crates/article_scraper/2.0.0-alpha.0 also has a port, but I haven't grasped how to get clean text content with an hour of fumbling, I'll spend more time and try to find out

Great work so far. I'll try some experimentation with these myself, too. Keep us posted on your progress!

I explored on 2 tracks:

manipulating the tag selection and content screening settings of the readability module of https://crates.io/crates/article_scraper/2.0.0-alpha.0
so that it can output more text than originally intended (just the main article body text)
it shall be a computationally less expensive approach if successful
it takes a lot of experiments to get workable settings

use https://github.com/rust-headless-chrome/rust-headless-chrome to convert the webpage to a long pdf page
then, use the pdftotext module of https://www.xpdfreader.com/ to convert the pdf to text
it's a hack approach, the result is good so far
story.verge.txt
story.verge.pdf
verge.txt
verge.pdf

my guess for its good result is:

when converting the webpage to pdf, the nesting of html DOM, etc. some layout information relevant to webpage rendering but not relevant to printed page layout is discarded, (hiding some text isn't the trick for a pdf document, right)
when converting pdf back to text, the pdftotext lib works with a "simplified" document and gets the only the text out

the draw back is:

convert webpage to pdf may fail
convert pdf to text may fail
conversion are more expensive

the unique value is:

when dealing with difficult webpages (abuse level use of js, challenging layout), an html parser will be beaten, the only way is to capture the page in some way and then parse it
there are projects that capture the webpage as an image and then do ocr
here we capture the webpage as pdf and then extract text, it's lighter and equally useful is dealing with difficult webpages

@Cormanz I further experimented on capture-to-pdf-convert-to-text, see code: https://github.com/jaykchen/smartgpt/blob/main/src/plugins/browse/headless/mod.rs
don't know thoroughly about the current browse module, just inserted to existing function as following:
pub async fn browse_url(
ctx: &mut CommandContext,
args: Vec,
) -> Result<ScriptValue, Box> {
let browse_info = ctx.plugin_data.get_data("Browse")?;
let params: [(&str, &str); 0] = [];
let url: String = args.get(0).ok_or(BrowseNoArgError)?.clone().try_into()?;

// let body = invoke::<String>(
//     browse_info,
//     "browse",
//     BrowseRequest {
//         url: url.to_string(),
//         params: params
//             .iter()
//             .map(|el| (el.0.to_string(), el.1.to_string()))
//             .collect::<Vec<_>>(),
//     },
// )
// .await?;

let content = headless::get_content_headless(&url).await.unwrap();

It's just an attempt to run something.
Will further the adaptation of readability algo as main approach.

Wow, that's fascinating! Might be worth integrating into the native browse plugin itself.

@Cormanz I've just made a draft pr, the caveat is that it works only on Linux, don't know how I can make it accessible on other platforms.
using readability lib, it works well in a small number of webpages centered around one big body of text; the official version returns nothing or little strange information on other pages, my adapted version returns mixed bag of data, which is not dependable yet

Perhaps keep it as a separate drop in plugin for linux only? Windows users can always use docker if they wanna try it.

@philip06 found an alternative library, it's robust and it offers all platforms support, please give it a spin when you're available, thanks

@philip06 found an alternative library, it's robust and it offers all platforms support, please give it a spin when you're available, thanks

Sure i'll try it out on windows and WSL


[SuperEditor] Image issues (atomic commands)

@matthew-carroll I've found several image issues since the atomic commands branch was merged. All issues are present on Android (I will test iOS shortly), and one is present on Mac.
All of the following were run on the stable branch of the example app. The only change I made to the code was to add an icon button that calls the following:
  void _insertImage() {
    commonOps.insertImage('https://storage.googleapis.com/cms-storage-bucket/c823e53b3a1a7b0d36a9.png');
  }

1. When an image is first added, the carat doesn't move to the ParagraphNode inserted below the image
Present on Android
However, if the same image is added again, this doesn't occur. I'm assuming it has something to do with fetching the image from the network the first time.
https://github.com/superlistapp/super_editor/assets/6467808/146372b4-dd8a-4f09-91f3-647bcbb1db08
2. When the carat is to the right of an image and enter/return is pressed, new ParagraphNode's are inserted below (as expected), but the carat does not move
Present on Android
https://github.com/superlistapp/super_editor/assets/6467808/0c3cb5f3-dfb1-4dd6-add8-5e8ce7b6d0d5
3. Can't add an image in a non-empty paragraph after a previous image was removed
Present on Android and Mac
Add an image, remove it, then add it again where the carat now rests. The following error occurs:
════════ Exception caught by foundation library ════════════════════════════════
The following _TypeError was thrown while dispatching notifications for PausableValueNotifier<DocumentSelection?>:
type 'UpstreamDownstreamNodePosition' is not a subtype of type 'TextPosition' in type cast

When the exception was thrown, this was the stack
#0      SuperEditorState._updateComposerPreferencesAtSelection
super_editor.dart:468
#1      ChangeNotifier.notifyListeners
change_notifier.dart:403
#2      ValueNotifier.value=
change_notifier.dart:530
#3      PausableValueNotifier.resumeNotifications
pausable_value_notifier.dart:49
#4      MutableDocumentComposer.onTransactionEnd
document_composer.dart:156
#5      Editor.execute
editor.dart:164
#6      CommonEditorOperations._insertBlockLevelContent
common_editor_operations.dart:1787
#7      CommonEditorOperations.insertImage
common_editor_operations.dart:1711
#8      KeyboardEditingToolbar._insertImage
mobile_toolbar.dart:193
#9      _InkResponseState.handleTap
ink_well.dart:1154
#10     GestureRecognizer.invokeCallback
recognizer.dart:275
#11     TapGestureRecognizer.handleTapUp
tap.dart:654
#12     BaseTapGestureRecognizer._checkUp
tap.dart:311
#13     BaseTapGestureRecognizer.acceptGesture
tap.dart:281
#14     GestureArenaManager.sweep
arena.dart:167
#15     GestureBinding.handleEvent
binding.dart:469
#16     GestureBinding.dispatchEvent
binding.dart:445
#17     RendererBinding.dispatchEvent
binding.dart:331
#18     GestureBinding._handlePointerEventImmediately
binding.dart:400
#19     GestureBinding.handlePointerEvent
binding.dart:363
#20     GestureBinding._flushPointerEventQueue
binding.dart:320
#21     GestureBinding._handlePointerDataPacket
binding.dart:293
#22     _invoke1 (dart:ui/hooks.dart:158:13)
#23     PlatformDispatcher._dispatchPointerDataPacket (dart:ui/platform_dispatcher.dart:382:7)
#24     _dispatchPointerDataPacket (dart:ui/hooks.dart:91:31)
The PausableValueNotifier<DocumentSelection?> sending notification was: PausableValueNotifier<DocumentSelection?>#9983b([DocumentSelection] -
  base: ([DocumentPosition] - node: "44a3dbf9-65c6-4068-98dd-a585c901ea50", position: ([UpstreamDownstreamNodePosition] - TextAffinity.downstream)),
  extent: ([DocumentPosition] - node: "44a3dbf9-65c6-4068-98dd-a585c901ea50", position: ([UpstreamDownstreamNodePosition] - TextAffinity.downstream)))
════════════════════════════════════════════════════════════════════════════════

https://github.com/superlistapp/super_editor/assets/6467808/9965ccb2-d20c-4995-8755-defb9657af07
https://github.com/superlistapp/super_editor/assets/6467808/8dd8ad71-292e-4ea1-9f2a-c50266ebdfa7

@angelosilvestre some of this might be related to the caret position ticket you're already working on. After we get that fixed, then please take a look at these issues.

Update:

When an image is first added, the carat doesn't move to the ParagraphNode inserted below the image

This seems to be happening because, after being fetched from the network, the image is being resized, and this doesn't cause the editor to rebuild. We probably want to integrate ContentLayers in the mobile interactors. There is an open ticket to refactor the mobile widget tree structure: https://github.com/superlistapp/super_editor/issues/647
I was able to fix the other two issues. We have tests for image insertion, I'll investigate to see why these tests didn't fail.


Mark user message as required

I found a conflict with maxResponseTokens @transitive-bullshit
Problem 1
From one side, buildMessages() now has logic to limit prompt length, because we include "history messages" and wanna know when to stop adding them. From another hand, prompt length defines a length of response, because total sub should not exceed maxModelTokens. The we have the following:

I don't send parentMessageIds in my chatbot, so my users send short prompts, but get responses only of 1000 tokens length, because it's a default value
I set maxResponseTokens: 4096 as max allowed value, expect maximum length responses
But the current logic filters out user prompt, so we send just systemMessage to ChatGPT that has no sense and is confusing to user
I can reduce maxResponseTokens, but don't know for how much because I want to use full capacity of 4096 tokens but don't know what user prompt length is

Solution
If user passed a prompt and it exceeds the limits, we should throw exception to him, instead of skipping the userPrompt and sending systemMessage only. Then let's mark systemMessage and userPromp as "required" messages, which means that if they are passed the take prioritized tokens space. And if they exceed limits - we throw exception.
Now the flow looks like this:

Validate systemMessage and userPromp, based on maxModelTokens - 1 value, ignoring maxResponseTokens. Where 1 - minimum space for response. If user enters maxModelTokens - 1 tokens length prompt, he will get 1 token in
response. If he enters prompt bigger than maxModelTokens - 1, there will be an exception
Validate historyPrompts, based on maxModelTokens - maxResponseTokens value
Send calculated size for response, based on numTokens and maxResponseTokens

This project is undergoing a major revamp; closing out old PRs as part of the prep process. Sorry! This was a good change, but the chatgpt package is pretty outdated at this point.


Update: Replace screenshot with GIF in Animate images with image overlay

Please refer to common-samples/issues/2787 for details.
James from Qt team mentioned that it might be more visually appealing to use a GIF instead of static image for this sample. Java is already using GIF and Qt, .NET and iOS are planning to use GIF. Android opts out because they use the screenshot in the app and GIF isn't supported.
The URL to the branch: https://github.com/Esri/arcgis-runtime-samples-ios/tree/Ting/Update-GIF-vNext/arcgis-ios-sdk-samples/Scenes/Animate images with image overlay
This is how GIF looks like on the developer website: https://developers.arcgis.com/java/sample-code/animate-images-with-image-overlay/

Didn't we have gifs before and removed them? I seem to remember something about the developer site not supporting them. Am I remembering that incorrectly? Or has that changed?

Didn't we have gifs before and removed them?

This is the only other commit I can find that has "gif" in its message https://github.com/Esri/arcgis-runtime-samples-ios/commit/f43a01cd999d785f3e0e0153fcd377fac0494502 using git log --oneline | grep gif, so I'm not sure about that.
Since I started to maintain this repo, all the screenshots are unified to use PNG. Yesterday I asked Mike and Trevor if it's appropriate to follow the suggestion in common-samples/issues/2787 to introduce GIF, and they said yes. So I made the change in #1081 to allow GIF.

... the developer site not supporting them. ... has that changed?

Please see in the description atop. Java used GIF and it seems to render fine on the new developer website. I think we only use the screenshot in the sample README and on the developer site, so it should be fine?
Android doesn't support GIF because their sample viewer pull in the screenshot as the collection cell background image and GIF cannot be displayed.

The gif looks great, but it is rather large (3.44 MB), especially compared to the previous screenshot. Gifs will understandably be larger, but I'm wondering if there's a way to reduce the size at all. Maybe it could it be shortened?

The gif looks great, but it is rather large (3.44 MB), especially compared to the previous screenshot. Gifs will understandably be larger, but I'm wondering if there's a way to reduce the size at all. Maybe it could it be shortened?

Good point. It is pretty large because it is 250 frames @ 50fps with a 64-color palette. I'll try with ezgif.com and gifsicle to shrink its size.

It's now working correctly on the 100.11.2 build of the doc website: https://developers.arcgis.com/ios/swift/sample-code/animate-images-with-image-overlay/


Storage backend ideas

Consider using Victoria Metrics, which supports promql, which may yield simpler integration with Grafana.

Thanos would allow for metric aggregation across clusters/environments: https://docs.bitnami.com/tutorials/create-multi-cluster-monitoring-dashboard-thanos-grafana-prometheus/#step-1-install-the-prometheus-operator-on-each-cluster


Remove hardcoded references to English and Swedish locales

Language strings are stored in separate general JSON files but the frontend contains a few hardcoded references which prevents other languages than English or Swedish from being used. This issue pertains to removing such hardcoded references so an arbitrary language translation can be used.

The metadata is bilingual, supporting a "local" language (the other one assumed to be English). To my knowledge the metadata does not specify its "local" language, so this information can be added to a baseline. Coupled with issue #19 this would specify a full locale (language and country), which can be useful. (and a single locale string could be used). Then the regular language selection logic could be used (if the browser's language is set to the metadata local one, use that, otherwise resort to English).


[4.11.0] ERROR Error: Uncaught (in promise): Error: Cannot match any routes.

https://forum.ionicframework.com/t/ionic-4-tabs-navigation-post-login-solution/143351
i cannot make this work, i think lazy loading feature doesn't cost that breaking changes with tabs every1 can not fix, now need another breaking change to make old tabs work
my package.json deps
"dependencies": {
"@angular/common": "7.2.8",
"@angular/core": "7.2.8",
"@angular/forms": "7.2.8",
"@angular/http": "7.2.8",
"@angular/platform-browser": "7.2.8",
"@angular/platform-browser-dynamic": "7.2.8",
"@angular/router": "7.2.8",
"@ionic-native/core": "^5.0.0",
"@ionic-native/splash-screen": "^5.0.0",
"@ionic-native/status-bar": "^5.0.0",
"@ionic/angular": "^4.1.1",
"cordova-android": "8.0.0",
"cordova-ios": "^5.0.0",
"cordova-plugin-device": "^2.0.2",
"cordova-plugin-ionic-keyboard": "^2.1.3",
"cordova-plugin-ionic-webview": "^4.0.0",
"cordova-plugin-splashscreen": "^5.0.2",
"cordova-plugin-statusbar": "^2.4.2",
"cordova-plugin-whitelist": "^1.3.3",
"core-js": "^2.5.4",
"rxjs": "~6.4.0",
"zone.js": "~0.8.29"
},
"devDependencies": {
"@angular-devkit/architect": "~0.13.5",
"@angular-devkit/build-angular": "~0.13.0",
"@angular-devkit/core": "~7.3.5",
"@angular-devkit/schematics": "~7.3.5",
"@angular/cli": "~7.3.1",
"@angular/compiler": "~7.2.2",
"@angular/compiler-cli": "~7.2.2",
"@angular/language-service": "~7.2.2",
"@ionic/angular-toolkit": "~1.4.0",
"@types/node": "~11.11.0",
"@types/jasmine": "~3.3.9",
"@types/jasminewd2": "~2.0.3",
"codelyzer": "~4.5.0",
"jasmine-core": "~3.3.0",
"jasmine-spec-reporter": "~4.2.1",
"karma": "~4.0.1",
"karma-chrome-launcher": "~2.2.0",
"karma-coverage-istanbul-reporter": "~2.0.1",
"karma-jasmine": "~2.0.1",
"karma-jasmine-html-reporter": "^1.4.0",
"protractor": "~5.4.0",
"ts-node": "~8.0.0",
"tslint": "~5.13.1",
"typescript": "~3.3.3333"
}

nvm, fixed it, but I can imagine how difficult it will look like when you have to add authguar and skiplocationchange
for those who will face same issue thats app.routing.module.ts
{
path: 'login',
loadChildren: './login/login.module#LoginPageModule'
},
{
path: 'tabs',
loadChildren: './tabs/tabs.module#TabsPageModule'
},
{ path: '', redirectTo: 'login', pathMatch:'full'},
{ path: '', loadChildren: './tabs/tabs.module#TabsPageModule' },
];


Converting indented comments

When you comment lines of code in Jupyter using a keyboard shortcut the # gets added at the beginning of the line instead of the beginning of indent, e.g.:
if True:
    print("not commented")
#    print("commented in Jupyter")
    # print("commented in .py file e.g. in PyCharm")

Then after we format it with black_nbconvert it looks like following:
if True:
    print("not commented")
    #    print("commented in Jupyter")
    # print("commented in .py file e.g. in PyCharm")

Maybe a better approach would be to check if there is more than 1 space between # and the start of comment text and then first take those spaces out before the # and add a single space between # and the start of comment text.

Thanks!
I don't think we would want to do something like that in this project, because its a very lightweight wrapper of black. Some editors also have the same behavior and applying black to those scripts would produce the same problematic results. But, you could also end up in situations where you don't want the solution that you propose (e.g.):
# for i in range(10):
#     print(i)

I think that supporting all the possible use cases is beyond the scope here, but you could open a feature request on one of the Jupyter repos (to change the notebook behavior) or on black (to add the feature that you're suggesting here).


Add Elastic Beanstalk deployment

Can copy Schema Guru's

Can add a script create-beanstalk-jar.sh, contents:
#!/bin/sh
rm -rf target
sbt assembly
tail -c +44 `ls target/scala-2.10/iglu-server-*` > iglu-server.jar

Originally this was going into Release 3, but pushing back as unclear what value this script adds...


📝Remove fetch as an entry point to main typedocs

Motivation
fetch() and its associated APIs are now part of the main effection module and as such, @effection/fetch is more of a private API and should not appear as part of the typedocs since there's no guarantee that it won't be moved somewhere else at any point.
Approach
remove it from the sidebar.

In #544 I went the other way and exposed all of the packages. I find the docs of the subpackages to be much easier to read, and it's easier to see which parts belong together.

In #544 I went the other way and exposed all of the packages. I find the docs of the subpackages to be much easier to read, and it's easier to see which parts belong together.

I can see that. I guess what I’m trying to discourage is folks thinking about the npm packages themselves as a public API, but I agree grouping them into categories of usage is super helpful. Is there any way to control the labels on the sidebar?

Is there any way to control the labels on the sidebar?

Maybe. TypeDoc has a category feature, but IMO it does not work very well, and last I checked it does not apply the categories in the sidebar at all.

I suppose we can leave it then. I just don't want anyone to be tempted to import { Stream } from '@effection/stream' that way, when we move things around internally like we did when we moved streams out of subscription, we're free to do so without yanking the carpet out from underneath someone.


Ethernet in esp32

I'm unable to initiate ethernet in esp32 using example code.
I'm using platformIO ESP-IDF sdk for testing Ethernet in esp32.
please guide me through this:
my test code is
/* ethernet Example

   This example code is in the Public Domain (or CC0 licensed, at your option.)

   Unless required by applicable law or agreed to in writing, this
   software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
   CONDITIONS OF ANY KIND, either express or implied.
*/
#include <stdio.h>
#include <string.h>

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

#include "esp_system.h"
#include "esp_err.h"
#include "esp_event_loop.h"
#include "esp_event.h"
#include "esp_attr.h"
#include "esp_log.h"
#include "esp_eth.h"

#include "rom/ets_sys.h"
#include "rom/gpio.h"

#include "soc/dport_reg.h"
#include "soc/io_mux_reg.h"
#include "soc/rtc_cntl_reg.h"
#include "soc/gpio_reg.h"
#include "soc/gpio_sig_map.h"

#include "tcpip_adapter.h"
#include "nvs_flash.h"
#include "driver/gpio.h"

#include "soc/emac_ex_reg.h"
#include "driver/periph_ctrl.h"

#include "../../lib/debug/debug.h"
#include "eth_phy/phy_lan8720.h"

#define DEFAULT_ETHERNET_PHY_CONFIG phy_lan8720_default_ethernet_config

#define CONFIG_PHY_ADDRESS  0
#define PIN_SMI_MDC         23
#define PIN_SMI_MDIO        18

static void eth_task(void *pvParameter);
static void eth_gpio_config_rmii(void);
static void phy_device_power_enable_via_gpio(bool enable);

void app_main(void)
{
	esp_err_t ret = ESP_OK;
	tcpip_adapter_init();
	esp_event_loop_init(NULL, NULL);

	eth_config_t config; // = DEFAULT_ETHERNET_PHY_CONFIG;
	memcpy(&config, &DEFAULT_ETHERNET_PHY_CONFIG, sizeof(eth_config_t));
	/* Set the PHY address in the example configuration */
	config.phy_addr = CONFIG_PHY_ADDRESS;
	config.gpio_config = eth_gpio_config_rmii;
	config.tcpip_input = tcpip_adapter_eth_input;
	config.phy_power_enable = phy_device_power_enable_via_gpio;

#ifdef CONFIG_PHY_USE_POWER_PIN
	/* Replace the default 'power enable' function with an example-specific
	   one that toggles a power GPIO. */
	config.phy_power_enable = phy_device_power_enable_via_gpio;
#endif

	ret = esp_eth_init(&config);

	char * prnt = ret == ESP_OK ? "\nesp_eth_init success.\n" : "\nesp_eth_init failed.\n";
	TRACE("%s", prnt);

	if(ret == ESP_OK)
	{
		esp_eth_enable();
		xTaskCreate(eth_task, "eth_task", 2048, NULL, (tskIDLE_PRIORITY + 2), NULL);
	}

}

static void eth_task(void *pvParameter)
{
	tcpip_adapter_ip_info_t ip;
	memset(&ip, 0, sizeof(tcpip_adapter_ip_info_t));
	vTaskDelay(2000 / portTICK_PERIOD_MS);

	while (1)
	{
		vTaskDelay(2000 / portTICK_PERIOD_MS);

		if (tcpip_adapter_get_ip_info(ESP_IF_ETH, &ip) == ESP_OK)
		{
			TRACE("\n~~~~~~~~~~~");
			TRACE("ETHIP:"IPSTR, IP2STR(&ip.ip));
			TRACE("ETHPMASK:"IPSTR, IP2STR(&ip.netmask));
			TRACE("ETHPGW:"IPSTR, IP2STR(&ip.gw));
			TRACE("~~~~~~~~~~~");
		}
	}
}

static void eth_gpio_config_rmii(void)
{
	// RMII data pins are fixed:
	// TXD0 = GPIO19
	// TXD1 = GPIO22
	// TX_EN = GPIO21
	// RXD0 = GPIO25
	// RXD1 = GPIO26
	// CLK == GPIO0
	phy_rmii_configure_data_interface_pins();
	// MDC is GPIO 23, MDIO is GPIO 18
	phy_rmii_smi_configure_pins(PIN_SMI_MDC, PIN_SMI_MDIO);
}

static void phy_device_power_enable_via_gpio(bool enable)
{

}

#ifdef CONFIG_PHY_USE_POWER_PIN
/* This replaces the default PHY power on/off function with one that
   also uses a GPIO for power on/off.

   If this GPIO is not connected on your device (and PHY is always powered), you can use the default PHY-specific power
   on/off function rather than overriding with this one.
*/
static void phy_device_power_enable_via_gpio(bool enable)
{
	assert(DEFAULT_ETHERNET_PHY_CONFIG.phy_power_enable);

	if (!enable) {
		/* Do the PHY-specific power_enable(false) function before powering down */
		DEFAULT_ETHERNET_PHY_CONFIG.phy_power_enable(false);
	}

	gpio_pad_select_gpio(PIN_PHY_POWER);
	gpio_set_direction(PIN_PHY_POWER,GPIO_MODE_OUTPUT);
	if(enable == true) {
		gpio_set_level(PIN_PHY_POWER, 1);
		ESP_LOGD(TAG, "phy_device_power_enable(TRUE)");
	} else {
		gpio_set_level(PIN_PHY_POWER, 0);
		ESP_LOGD(TAG, "power_enable(FALSE)");
	}

	// Allow the power up/down to take effect, min 300us
	vTaskDelay(1);

	if (enable) {
		/* Run the PHY-specific power on operations now the PHY has power */
		DEFAULT_ETHERNET_PHY_CONFIG.phy_power_enable(true);
	}
}
#endif

output is:
16:42:15.736> ets Jun  8 2016 00:22:57
16:42:15.736>
16:42:15.736> rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
16:42:15.736> configsip: 0, SPIWP:0xee
16:42:15.736> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
16:42:15.736> mode:DIO, clock div:2
16:42:15.736> load:0x3fff0018,len:4
16:42:15.736> load:0x3fff001c,len:5528
16:42:15.736> load:0x40078000,len:0
16:42:15.736> ho 12 tail 0 room 4
16:42:15.736> load:0x40078000,len:12488
16:42:15.736> entry 0x40078fd4
16:42:15.736> I (30) boot: ESP-IDF 2.300.0 2nd stage bootloader
16:42:15.736> I (30) boot: compile time 14:53:41
16:42:15.736> I (30) boot: Enabling RNG early entropy source...
16:42:15.736> I (35) boot: SPI Speed      : 40MHz
16:42:15.736> I (39) boot: SPI Mode       : DIO
16:42:15.736> I (43) boot: SPI Flash Size : 4MB
16:42:15.736> I (47) boot: Partition Table:
16:42:15.736> I (50) boot: ## Label            Usage          Type ST Offset   Length
16:42:15.736> I (58) boot:  0 nvs              WiFi data        01 02 00009000 00006000
16:42:15.736> I (65) boot:  1 phy_init         RF data          01 01 0000f000 00001000
16:42:15.736> I (73) boot:  2 factory          factory app      00 00 00010000 00100000
16:42:15.736> I (80) boot: End of partition table
16:42:15.736> I (84) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x0ad38 ( 44344) map
16:42:15.736> I (109) esp_image: segment 1: paddr=0x0001ad60 vaddr=0x3ffc0000 size=0x01f14 (  7956) load
16:42:15.736> I (112) esp_image: segment 2: paddr=0x0001cc7c vaddr=0x40080000 size=0x00400 (  1024) load
16:42:15.736> I (116) esp_image: segment 3: paddr=0x0001d084 vaddr=0x40080400 size=0x02f8c ( 12172) load
16:42:15.736> I (129) esp_image: segment 4: paddr=0x00020018 vaddr=0x400d0018 size=0x25d58 (154968) map
16:42:16.339> I (187) esp_image: segment 5: paddr=0x00045d78 vaddr=0x4008338c size=0x04a6c ( 19052) load
16:42:16.339> I (195) esp_image: segment 6: paddr=0x0004a7ec vaddr=0x400c0000 size=0x00000 (     0) load
16:42:16.339> I (200) boot: Loaded app from partition at offset 0x10000
16:42:16.339> I (201) boot: Disabling RNG early entropy source...
16:42:16.339> I (206) cpu_start: Pro cpu up.
16:42:16.339> I (209) cpu_start: Starting app cpu, entry point is 0x40080e74
16:42:16.339> I (0) cpu_start: App cpu up.
16:42:16.339> I (220) heap_init: Initializing. RAM available for dynamic allocation:
16:42:16.339> I (227) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
16:42:16.339> I (233) heap_init: At 3FFCB790 len 00014870 (82 KiB): DRAM
16:42:16.339> I (239) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
16:42:16.339> I (245) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
16:42:16.339> I (252) heap_init: At 40087DF8 len 00018208 (96 KiB): IRAM
16:42:16.339> I (258) cpu_start: Pro cpu start user code
16:42:16.339> I (276) cpu_start: Starting scheduler on PRO CPU.
16:42:16.339> I (0) cpu_start: Starting scheduler on APP CPU.
16:42:16.339> I (463) emac: mac version 1137a
16:42:16.339> I (463) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
16:42:16.339>
16:42:16.339> esp_eth_init success.
16:42:16.339>
16:42:16.339> I (473) emac: emac start !!!
16:42:16.339>
16:42:16.339> I (473) emac: emac reset done
16:42:17.092> E (1473) emac: Timed out waiting for PHY register 0x2 to have value 0x0007 (mask 0xffff). Current value 0xffff
16:42:17.984> E (2473) emac: Timed out waiting for PHY register 0x3 to have value 0xc0f0 (mask 0xfff0). Current value 0xffff
16:42:18.983> E (3473) emac: Timed out waiting for PHY register 0x2 to have value 0x0007 (mask 0xffff). Current value 0xffff
16:42:19.983> E (4473) emac: Timed out waiting for PHY register 0x3 to have value 0xc0f0 (mask 0xfff0). Current value 0xffff
16:42:20.983> E (5473) emac: Timed out waiting for PHY register 0x2 to have value 0x0007 (mask 0xffff). Current value 0xffff
16:42:21.985> E (6473) emac: Timed out waiting for PHY register 0x3 to have value 0xc0f0 (mask 0xfff0). Current value 0xffff
16:42:22.983> E (7473) emac: Timed out waiting for PHY register 0x2 to have value 0x0007 (mask 0xffff). Current value 0xffff

You didn't say anything about hardware or if you verified phy address setting

My hardware is ESP32-WROM-32
Interfaced with LAN8720  (MDIO-18, MDC-23)
Phyter address is 0.
schematic:

The problem could be the 74VC04 inverter in your clock signal.

Dear all,
The above issue has been solved (solution: phyter address = 1).
But Now my device is unable to get IP.
Is there any thing else I've to add to get IP like dhcp_start() or something ?

I removed 74VC04 and and changed R83 to 0 ohm and shorted. Also changed GPIO0 pull up from 330K to 4.7K.
Still no IP.
I'm unable to find what is the problem.
Please help me !!!

This is what I am using and it works just fine. Code is not cleaned up and used for something else to prototype with, but maybe there is something in there that may help.

main.zip

@KrrishnaShah: Your schematic does not show a critical part: Is LED2/nINTSEL pulled low so that nINT/
REFCLKO is configured as clock output? (e.g. is the LED connected to GND or VDD?)
From the data sheet:

When nINTSEL is floated or pulled toVDD2A, nINT is selected for operation on the nINT/REFCLKO pin (default).
• When nINTSEL is pulled low to VSS, REFCLKO is selected for operation on the nINT/REFCLKO pin.

@sauttefk I wanted to know this also. It appears that REFCLKO or nINT is not used as the same clock from the crystal goes to the esp32 in the GPIO0 pin.
Another thing that I'm corious: for what porpoise there are 2 33Ohms resistors in series with the CLKIN pin? Is something to do with high speed, impedance matching or signal isolation?

@sauttefk My phyter LED schematic is as :

Thanks everyone
The issue has been solved.
Problem:  was the interference in clock and signals ,
Solution: Track redesign and done


PR: Disable zooming and update displayed scaling percent when "Fits plot to window" is checked

Description of Changes

[x] Wrote at least one-line docstrings (for any new functions)
[x] Added unit test(s) covering the changes (if testable)
[x] Included a screenshot or animation (if affecting the UI, see Licecap)

Issue(s) Resolved
Fixes #9405
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.
I certify the above statement is true and correct: Jean-Sébastien Gosselin

Hello @jnsebgosselin! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file spyder/plugins/plots/widgets/tests/test_plots_widgets.py:

Line 421:1: W293 blank line contains whitespace

Awesome @jnsebgosselin !
There is the pepspeaks issue?

In the file spyder/plugins/plots/widgets/tests/test_plots_widgets.py:

Thoughts on https://github.com/spyder-ide/spyder/issues/9367 :-p ?


Bad emit for private static class fields

Describe the bug
We using the emitter in Deno, static private fields are being emitted without their static modifier.
Input code
class A {
  static log() {
    console.log(this.#A);
  }
  static #A = "a";
}

A.log();

Expected behavior
Expect the following to be emitted:
class A {
  static log() {
    console.log(this.#A);
  }
  static #A = "a";
}

A.log();

What actually gets emitted is:
class A {
    static log() {
        console.log(this.#A);
    }
    #A = "a";
}
A.log();

Version
swc_bundler = "0.45.0"
swc_common = "0.10.21"
swc_ecmascript = "0.45.0"
Additional context
Ref: denoland/deno#11287

Awesome, thank you!


W18: Invalid character in group name error

Hi, I am using this colorscheme. I yesterday installed Kali Linux and it has neovim version 0.7.2 r-3 but this colorscheme gives that error in kali. but it does not give that error to me in Alpine, Arch and Artix which uses neovim version 0.8.1 .. in the readme i just check here it says nvim version 0.6 or greater. so please help.

@notwidow Looks like this is the problem discussed in #79. I had this issue and upgrading to version >= 0.8 like they advise fixed it for me.


Compatibility issue Angular 6

I get following error after upgrading my project to Angular 6:
Probably some tweaks that still have to be done to be compatible with 6?
Any Idea when this will be available?
Can't bind to 'ngOutletContext' since it isn't a known property of 'ng-container'.
1. If 'ngOutletContext' is an Angular directive, then add 'CommonModule' to the<EMAIL_ADDRESS>of this component.
2. To allow any property add 'NO_ERRORS_SCHEMA' to the<EMAIL_ADDRESS>of this component. (")="onMouseOver(i)" (mouseenter)="onHovering($event)"><ng-container [ngTemplateOutlet]="templateRef" [ERROR ->][ngOutletContext]="{$item:item, $entity: item.entity, $id:item.id, $index:i}"></ng-container><ng-cont"): ng:///IqSelect2Module/IqSelect2ResultsComponent.html@0:417
    at syntaxError (compiler.js:215)
    at<EMAIL_ADDRESS>(compiler.js:14705)
    at<EMAIL_ADDRESS>(compiler.js:22712)
    at<EMAIL_ADDRESS>(compiler.js:22699)
    at compiler.js:22642
    at Set.forEach (<anonymous>)
    at<EMAIL_ADDRESS>(compiler.js:22642)
    at compiler.js:22552
    at Object.then (compiler.js:206)
    at<EMAIL_ADDRESS>(compiler.js:22551)
compiler.js:215 Uncaught Error: Template parse errors:

Hi! that looks more like an Angular 4 to Angular 5 migration issue to me. That was modified some time ago to keep the compatibility with newer angular versions.
I think you may have an old version of the component installed. FYI, the last one to support bootstrap 3 was version 7, and starting from version 8, the component is based on bootstrap 4.
Don't know what version you are using, but probably making sure you are on 7 or 8 will fix that issue.

Hmm, I'm pretty sure i'm running Angular 6, and I have the latest version of this component.
Maybe there is there another way to update it? I'm not seeing a version 7 ... or 8.
It would be of great help!

Aha, I stand corrected. I was able to install the version 8. I just didn't find it in the releases tab at github before, that's why. Maybe that's a good place to document it?
I confirm it works with A6, supporting Bootstrap4!


[gha] single job if needed for bors

Motivation
GHA doesn't do a good job of providing the identity of a job that has succeeded/failed.   Less so for a workflow file.
In order for bors to have something to hold on to, creating a success job that'll fast fail if any job fails.
So this was a bit tricky.   The result from a job is a string of "failure","skipped","success" or "cancelled", see:
https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#needs-context
Fun to figure out.
Also if a needed job was skipped by default, the dependent job is skipped as well (hence the "always()" in the if line of the job).  Commented inline.
Need to test this by creating failure cases and testing, as well as success case.  Created a PR for each and tested there:
Success PR: https://github.com/diem/diem/actions/runs/494420181 -- notice it works with skips as well.
And failure PR:  https://github.com/diem/diem/actions/runs/494729561.
This is as easy as gha can make it for bors.
Have you read the [Contributing Guidelines on pull requests]
Y
Test Plan
Two draft PRs one forcing success and another failure.  Feel free to inspect, will close when merged/cancelled.
https://github.com/diem/diem/pull/7267
https://github.com/diem/diem/pull/7266
Related PRs
https://github.com/diem/diem/pull/7267
https://github.com/diem/diem/pull/7266

Eh, not the best.   The success job doesn't get marked as skipped when the first failure of a job it depends occurs.
Maybe also need the tear down step to emit a job failure message in some secure way?

Eh, not the best.   The success job doesn't get marked as skipped when the first failure of a job it depends occurs.
Maybe also need the tear down step to emit a job failure message in some secure way?

If we decide to use this anyway, drop last commit before merging.

If we decide to use this anyway, drop last commit before merging.

/land

/land


rewrite installation guide

As many users are reporting installation failures, I reorganized the Installation Guide.
Major changes compared to the current one is to recommend use of wheels as the first choice.

Could someone take a look on this?

Added some comments on the grammar but I think it looks good otherwise. Could you also take a look at the CI failure?

Jenkins, test this please.

Thanks for review! Fixed accordingly.

Thanks for the quick fixes. However, CI is still complaining, do you mind taking another look?

Thank you, it should be fixed now...

LGTM!


Jenkins Octopus Plugin does not handle release with version template

The Jenkins plugin fails to generate a build summary when the release number is templated. For example 1.0.i:

See https://octopus.zendesk.com/agent/tickets/67925 and https://help.octopus.com/t/jenkins-plugin-failed-to-generate-build-summary/26552

another customer with this issue - https://octopus.zendesk.com/agent/tickets/85534

Also me, and this person : https://help.octopus.com/t/octopus-deploy-jenkins-plugin-issue/29366


Unable to deploy successfully, github pages does not support jekyll scholar plugin.

Thank you so much for this beautiful template! I have my site all set up (based somewhat off of your group's site and it runs locally just fine, but the GH Pages deployment is breaking because it doesn't recognize the bibliography command, which is because jekyll-scholar isn't in GitHub's supported plugins. I tried a bunch of solutions to this that I found in a number of places, but haven't quite gotten it working (the closest attempt did publish the site, but seemed to just be "raw" HTML (without css).
I'm really not an expert in this stuff. Can you tell me in detail how to deploy this blog on github pages?
Thanks a million!

Did you follow the directions? You need to use rake and have master and source branches.

Thanks for the reply, I have redone a template repository following your instructions. However, my site is still not rendering the CSS. Can you help me check where the error is? My repository: https://github.com/wangyb97/aibio-lab. My website: https://wangyb97.github.io/aibio-lab/

Unfortunately I do not have the time to do tech support. Fortunately Jekyll is very popular and this website template does not involve anything out of the ordinary. Closing as I cannot troubleshoot the issue.


Include request ID in debug calls

That way it's possible to narrow debugging per specific request. Useful when debugging parallel requests.
How

Generate a reproducible request ID for req.url and the timestamp. Do this as soon as the all the prerequisites are known, so debugging is consistent and tight to the request ID through the library's execution.
Create a custom debug() instance and use it everywhere.

I've decided not to generate a request ID because it would make little sense. Random ID would be, well, random between calls, which loses the connection between the same request when debugging by iterating. Static ID is the same as request URL, so instead I've added request method and URL into the debug calls.


Remove dependency on is-buffer

As the function was used only once, I inlined it in the code and removed the dependency on is-buffer from package.json. Your package is now dependency-free!

Test failed :(

Added with https://github.com/hughsk/flat/pull/100


Add (1) information on browser navigation commands along with a corresponding link, and (2) code which enables page jumps; remove unnecessary link segments

Add (1) information on browser navigation commands along with a corresponding link, and (2) code which enables page jumps; remove unnecessary link segments

Headers have IDs added by https://github.com/SeleniumHQ/docs/blob/gh-pages/docs.js#L23-L40, so these patches are unnecessary.


Delete StandardCharsets from codebase

Summary:
StandardCharsets was deprecated and marked for removal, since 0.73 was released we can now safely delete it
changelog: [Android][Deleted] Delete StandardCharsets class, please use java.nio.charset.StandardCharsets instead
Differential Revision: D47306709

Closing as abandoned internally


Error parsing this YAML file.

Hi,
As a pre-commit hook I'm checking to see that my code is able to be parsed, like so:
for file in `git diff-index --cached --diff-filter=AM --name-only HEAD`
do
  echo "Validating ${file}..."

  case "${file##*.}" in
    "pp")  puppet parser validate ${file}
           ;;

    "erb") /opt/puppet/bin/erb -P -x -T '-' ${file} | /opt/puppet/bin/ruby -c >/dev/null
           ;;

    "yaml") /opt/puppet/bin/ruby -ryaml -e "YAML.load_file '${file}'" >/dev/null
           ;;

  esac

  EXITCODE=$((EXITCODE + $?))
done

This file causes the pre-commit hook to fail with the following error:
root@klynton:puppetcode # git commit -m 'really'
Validating modules/staging/spec/fixtures/hiera.yaml...
/opt/puppet/lib/ruby/1.9.1/psych.rb:203:in `parse': (modules/staging/spec/fixtures/hiera.yaml): block sequence entries are not allowed in this context at line 2 column 12 (Psych::SyntaxError)
	from /opt/puppet/lib/ruby/1.9.1/psych.rb:203:in `parse_stream'
	from /opt/puppet/lib/ruby/1.9.1/psych.rb:151:in `parse'
	from /opt/puppet/lib/ruby/1.9.1/psych.rb:127:in `load'
	from /opt/puppet/lib/ruby/1.9.1/psych.rb:297:in `block in load_file'
	from /opt/puppet/lib/ruby/1.9.1/psych.rb:297:in `open'
	from /opt/puppet/lib/ruby/1.9.1/psych.rb:297:in `load_file'
	from -e:1:in `<main>'

################################################################
### Please fix the errors above before committing your code. ###
################################################################

root@klynton:puppetcode #

This simple change allows the YAML to be parsed.

This is legit. If you would add a test to our suite do the same validation you would be my hero.

@nibalizer Sent a PR to https://github.com/bodeco/bodeco_module_helper/pull/12 with the code.


Possible issues with timestamp column

This repo is a fantastic resource!
I noticed an issue with the match between time stamps and recorded years/months/days:
# Load data
con <- "https://github.com/AquaAuma/FishGlob_data/blob/main/outputs/Compiled_data/FishGlob_public_clean.RData?raw=true"
load(url(con))
# Compare time stamps and years for the first bunch of rows
data[1:10, c("timestamp", "year")]

> # A tibble: 10 × 4
   timestamp   year month   day
   <chr>      <int> <int> <int>
 1 2021-03-01  1983     8    18
 2 2021-03-01  1983     8    18
 3 2021-03-01  1983     8    18
 4 2021-03-01  1983     8    18
 5 2021-03-01  1983     8    18
 6 2021-03-01  1983     8    18
 7 2021-03-01  1983     8    18
 8 2021-03-01  1983     8    19
 9 2021-03-01  1983     8    19
10 2021-03-01  1983     8    19

I think the years/months/day columns are probably correct and the timestamp column has been incorrectly defined? The entities in that column vary in structure (e.g. yyyy-mm-dd, yyyy-mm and mm/yyyy).
I think at least some surveys record the time of day of each trawl deployment and/or retrieval as well. It might be useful to retain that information.

the timestamp column relates to the when the data were retrieved for creating the dataset, and it's not related to the sampling time. We'll try to reformat if enough time, but it's really just to indicate when the integration of datasets was done. This is implications for when we matched the surveys with the taxonomy for instance

Thanks for this clarification! That makes sense. If possible, you might want to consider renaming the timestamp column to make its meaning clearer & prevent accidental misuse (e.g., timestamp_data_retrieval). A separate column with the time stamps of surveys (year/month/day + time where available) could be useful.

since it's detailed in the manuscript in Table 2, I won't change the name


BUG[NAVIGATION PANEL] The 'Eco-news' page is shown in landing page after clicking on the 'Eco-advices' page

BUG[NAVIGATION PANEL] The 'Eco-news' page is shown in landing page after clicking on the 'Eco-advices' page
Environment:
OS: Windows 10Pro, V.1909
Browser: Chrome 80.0.3987.122 / 64-bit
Steps to reproduce:

Go to the site https://ita-social-projects.github.io/GreenCityClient/#/welcome
Click on the 'Eco-advices' page
Verify the opened page

Actual result:The 'Eco-news' page is shown in landing page after clicking on the 'Eco-advices' page
Expected result:The 'Eco-advices' page is shown in landing page after clicking on the 'Eco-advices' page

Can you to reproduce this bug?  Because after clicking on a link "Eco-Advices" I see scrolling to the Eco-Advices section
https://nimbusweb.me/s/share/3965993/mdqhgp8e9nvirgzu3tck

I always have random presence of this page:scrolling, displaying another page. Link for video
https://drive.google.com/open?id=1iDujX1y_aHWC5JexsSR2qhO0jwmXvJKn

Not reproducible


Support for Anthropic claude-3-5-sonnet

API Document give a full example in the config.yml

API Documentation
https://docs.anthropic.com/en/api/getting-started

Already supported. Available in the next release.
https://github.com/sigoden/aichat/blob/6d148c9c53cd4ac7849f52b956aba87e84cc3d4d/models.yaml#L348


Is there any way to name generations with a format (model used, gap fill technique) when created from the Depth tab?

I don't quite seem to be finding it if it exists. I'm sure there's some way to, but I'm not skilled enough to edit whatever settings necessary on my own.

Sadly, it is not possible to change as of now. https://github.com/thygate/stable-diffusion-webui-depthmap-script/issues/280

That makes sense. Perhaps in the future. Many thanks for the prompt response!


Tasks pause after system time is modified.

I do know that "Tasks pause after system time is modified. " is reasonable.
If there is a task which will be triggerd at 12:00:00 2019/3/27. After it is registered, we modify the system time to 2019/3/20. Obviously, the task will paused for a long time, it's OK.
But if we have a task which triggers once an hour, last time it triggered at 12:00:00 2019/3/26. Then it will be triggered at 13:00:00 2019/3/26, but if we modify the time to 2019/3/20 for some tests or just an accident, it will be paused for a few days.
Of course, the situation is reasonable. But for users, we don't really hope the task stopped for this kind  of sistuation. Cause we registered a simple LOOP job and hope it will trigger once an hour continually no matter it's year 1990 or 2020.
Firstly, I wanna know if Quartz has some supports for this kind of situations. And if not, would you please help me with the situation. We have requests to modify system time for some function, our plan is to refresh all the tasks every after the time modification. Then all the tasks will be triggerd properly just like they are registerd for the first time. Do you have some suggestions for me to handle the issue？

Hi, fire time of any trigger is depends on system time. If you go modified the system time while scheduler is running, then there is no guarantee of anything at all. If you must do this, then I would suggest you restart the scheduler after each system time has changed.


reflected XSS not registering on score board

Hello,
I thought I successfully performed a reflected XSS attack by visiting the following URL:
http://localhost:3000/#/search?q=<iframe src%3D"javascript:alert(`xss`)">
However, it is not showing up as completed on the score board.
If this is not enough to complete the challenge, maybe a note could be added to the book about what does qualify.
Thanks!

Hi! I just started my server and your link gives me exactly the expected solved notification... Small hint: "Where does everything after # in any URL never go?" ;-)


Added a Java Hello World for Hacktoberfest 2018

What the program do ?
Says Hello World in Java

In what programming language it is written?
Java

Yay!


Button to open the backup / restore directory (or any settable path)

What's your idea?
It would be convenient if there was a way to quickly open the path which is set for the backup or restore directory.
Instead of introducing an additional button for this function (if UI space is a concern), it may be nice for the folder icon button (which is used to set the backup / restore path) to double as the open-folder-in-file-manager button via the same Ctrl / Shift feature that the game row menu buttons use for their quick actions, such as Preview.
The above suggestion makes more sense if you consider extending this feature to all folder icons in Ludusavi, such as all those shown on the Custom Games and Other tabs. This way, any path that can be set via said icon could be quickly used to browse it in a system's file manager.
Thanks!
Ludusavi 0.19.0

I went with shift+click :D Demo:
https://github.com/mtkennerly/ludusavi/assets/4934192/f1d73418-aa8d-4c37-afbc-2c9e764e96f5

Very nice! Thanks for the addition 😄


Wildcard Let's Encrypt certs do not validate after DST Root CA expiration

Title: Wildcard Let's Encrypt certs do not validate after DST Root CA expiration
Description:
After one of the original Let's Encrypt CAs expired, wildcard certs issued by LE no longer validate with envoy.  I am not able to personally reproduce with non-wildcards.
I believe this is due to boringssl behavior, though I'd expect all LE certs have this problem in that case.
Repro steps:
To make a self contained reproduction, I've created an envoy configuration with an http listener (http://localhost:10000) that proxies to a local https listener (https://test-upstream.letsencrypt.localhost.pomerium.io:10443) that proxies the request to httpbin.org.  The https listener is using a wildcard let's encrypt cert.

Download gist.  The included certificates were generated for this repro and not used anywhere.
Run with envoy -c config.yaml
Verify system curl sees https://test-upstream.letsencrypt.localhost.pomerium.io:10443 as valid:

curl -I https://test-upstream.letsencrypt.localhost.pomerium.io:10443
HTTP/1.1 200 OK
date: Thu, 07 Oct 2021 20:56:19 GMT
content-type: text/html; charset=utf-8
content-length: 9593
server: envoy
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 21

Curl through envoy

curl localhost:10000
upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

You'll note curl has no issue but envoy itself does.
Admin and Stats Output:
n/a
Config:
#
# http://localhost:10000 -> [envoy] -> https://test-upstream.letsencrypt.localhost.pomerium.io:10443 -> [envoy] -> https://httpbin.org
#
# be sure to uncomment your operating system's cert store.

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      protocol: TCP
      address: <IP_ADDRESS>
      port_value: 9901
static_resources:
  listeners:
    - name: http
      address:
        socket_address:
          protocol: TCP
          address: <IP_ADDRESS>
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                access_log:
                - name: envoy.access_loggers.stdout
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                route_config:
                  name: test_upstream
                  virtual_hosts:
                    - name: test_upstream
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: test_upstream
                            auto_host_rewrite: true
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

    - name: https-upstream
      address:
        socket_address:
          protocol: TCP
          address: <IP_ADDRESS>
          port_value: 10443
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: upstream_https
                access_log:
                - name: envoy.access_loggers.stdout
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                route_config:
                  name: httpbin.org
                  virtual_hosts:
                    - name: httpbin.org
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: httpbin.org
                            auto_host_rewrite: true
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          transport_socket:
            name: envoy.transport_sockets.tls
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
              common_tls_context:
                tls_certificates:
                  certificate_chain:
                    filename: ./fullchain.pem
                  private_key:
                    filename: ./privkey.pem

  clusters:
    - name: test_upstream
      connect_timeout: 1s
      type: LOGICAL_DNS
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          common_tls_context:
            validation_context:
              trusted_ca:
                filename: /etc/ssl/cert.pem # osx
                # filename: /etc/ssl/certs/ca-certificates.crt # linux
      load_assignment:
        cluster_name: test_upstream
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: test-upstream.letsencrypt.localhost.pomerium.io
                      port_value: 10443

    - name: httpbin.org
      connect_timeout: 1s
      type: LOGICAL_DNS
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          common_tls_context:
      load_assignment:
        cluster_name: test_upstream
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: httpbin.org
                      port_value: 443

Logs:
[2021-10-07T20:49:18.754Z] "GET / HTTP/1.1" 503 UF 0 195 2 - "-" "curl/7.79.0" "7889b267-29b4-4680-930f-f20807690799" "localhost:10000" "<IP_ADDRESS>:10443"
[2021-10-07 16:48:39.667][12679903][debug][pool] [source/common/conn_pool/conn_pool_base.cc:407] [C1] client disconnected, failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
[2021-10-07 16:48:39.667][12679903][debug][router] [source/common/router/router.cc:1075] [C0][S10352822734306019930] upstream reset: reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
[2021-10-07 16:48:39.668][12679903][debug][http] [source/common/http/filter_manager.cc:935] [C0][S10352822734306019930] Sending local reply with details upstream_reset_before_response_started{connection failure,TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED}

i think i understand the related boringssl issue - but also hoping that the certs are being updated to remove the expired one in distros etc

@phlax the way the trust chain is built and evaluated is the problem.  On the same system, other tools (such as curl) work fine with the expired CA present.
You can see some discussion in the boringssl thread about the behavior.  I'm not sure if just pulling in the fix from Boring is sufficient but I'd expect the behavior to be to follow the shortest CA chain and validate that.  What's happening now is the longer of two trust chains is being followed and the root of it is expired, even though a shorter (and also trusted) root is present.
(you updated as I was responding)
Yes, the expired root cert will likely get removed over time but a fix within envoy/boring would be dramatically more timely.  Since most software isn't impacted, I don't think operating system vendors have a lot of pressure to remove the old CA.  Openssl, for instance, has the shorter chain evaluation behavior.
If envoy doesn't address this, folks will likely need to rebuild their system trust stores when deploying envoy outside of the official container, which doesn't scale well and will lead to a lot of user frustration.

yep, understood ive raised it with other maintainers - backporting to old versions is non-trivial - but as you say may well be necessary

Great thanks @phlax 👍 .

FWIW, Ubuntu already blacklisted the expired certificate (https://launchpad.net/ubuntu/+source/ca-certificates/20210119~18.04.2).

hi @travisgroth thanks for reporting this
im assuming that you are running with the envoyproxy/envoy docker image - is that correct ?
could you also let us know which version you are using (im hoping that the v1.20-latest image may have escaped this problem)

I can confirm the offending cert still exists in v1.20.0 docker image. Will there be new images built without that certificate, or an idea of when 1.21 is coming (presumably without this cert)? Right now we are building the following to work around the issue:
FROM envoyproxy/envoy:v1.20.0

USER root

RUN apt-get update \
    && apt-get install -y --no-install-recommends ca-certificates

Still an issue

As I understand it, the issue doesn't have anything to do with which CA certs are in the OS or docker image, it has to do with cross-signed certs with one of the CAs being expired. Correct?
@davidben It sounds like if Envoy sets X509_V_FLAG_TRUSTED_FIRST this should be resolved. Is that correct? What are the chances that setting that flag will break some other valid certificate chain is currently working?

@travisgroth if you want to help with a fix, any chance you can figure out how to add a new (self signed) cert to https://github.com/envoyproxy/envoy/blob/main/test/extensions/transport_sockets/tls/test_data/certs.sh that is valid but fails validation due to this bug?

BoringSSL flipped the default already.
https://boringssl.googlesource.com/boringssl/+/8f5eb80b810ff63d14ad3535cb16f7cb8271a4f5
Also it whether it may break other stuff, you'll just have to try it and see I'm afraid. The legacy verifier you're using doesn't have a real path builder. Trusted first merely changes it from one arbitrary decision to the other. It's possible to construct inputs that require it enabled, ones that require it disabled, and ones that don't work with either.
I think, on balance, it is better to have it enabled, since we know Let's Encrypt works better with it, and it matches what OpenSSL upstream does. But I can't give you anything beyond that. We don't use this code in Chrome and will be replacing it more broadly. The replacement won't look like this stack at all and will look closer to PR #10621. (Many of the problems with this stack, are, alas, part of the public API. I have hopes to eventually align behaviors and make this API a shim over the old one, but that's very speculative and may not even be feasible. It certainly has no future as the preferred API.)

I believe this is still an issue, and does not only affect wildcard certificates. I believe I hit the same issue using TLS termination in istio towards a site using a certificate signed by letsencrypt. I condensed it to a small testcase. With the below config, curl http://localhost:10000 yields stream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED. The log says:
[2022-02-25 14:11:55.027][15][debug][connection] [source/extensions/transport_sockets/tls/cert_validator/default_validator.cc:202] X509_verify_cert: certificate verification error at depth 3: certificate has expired
[2022-02-25 14:11:55.027][15][trace][connection] [source/extensions/transport_sockets/tls/ssl_handshaker.cc:52] [C3] ssl error occurred while read: SSL
[2022-02-25 14:11:55.027][15][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:225] [C3] TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
[2022-02-25 14:11:55.027][15][debug][connection] [source/common/network/connection_impl.cc:249] [C3] closing socket: 0
[2022-02-25 14:11:55.027][15][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:225] [C3] TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

Config:
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: <IP_ADDRESS>
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          http_filters:
          - name: envoy.filters.http.router
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  host_rewrite_literal: letsencrypt.org
                  cluster: letsencrypt.org
  clusters:
  - name: letsencrypt.org
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    load_assignment:
      cluster_name: letsencrypt.org
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: letsencrypt.org
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        sni: letsencrypt.org
        common_tls_context:
          validation_context:
            trusted_ca:
              filename: /etc/ssl/certs/ca-certificates.crt
            match_typed_subject_alt_names:
            - san_type: DNS
              matcher:
                exact: letsencrypt.org

The BoringSSL version was recently updated in Envoy, and I believe the new version should work for this case. Can someone validate against main?

The BoringSSL version was recently updated in Envoy, and I believe the new version should work for this case. Can someone validate against main?

I can confirm that it works in main. I downloaded the docker-distroless artifact from the last complete successful build and loaded it with "docker load" and added a new service to my test above docker-compose.yml which is the exact same except exposing envoy on port 10001. curl http://localhost:10000 (1.21-latest) yields a CERTIFICATE_VERIFY_FAILED error. curl http://localhost:10001 (20220321.5) successfully returns the index page of letsencrypt.org.


Content for What is Metadata module

CS 7810 Midterm. Alex Moore, Jehan Fernando, Chris Menart: Wrote 'What is Metadata?'

Can you please go back and more appropriately add in-line citations. Generally, citations follow text [x,y].  format. For example lines 27/28 could use more granular citing, or distinct citations.


[GLUTEN-1792][VL] Remove local cache files on Executor exit

What changes were proposed in this pull request?
Remove cache file on Executor exit to prevent disk full as Executor restarts.
Fixes: #1792
How was this patch tested?
unit tests

@zhouyuan how many files it needs to remove? If number is too large, let's move it to an async thread

@zhouyuan how many files it needs to remove? If number is too large, let's move it to an async thread

Basically, this is controlled by spark.gluten.sql.columnar.backend.velox.ssdCacheShards, we won't have too many files to delete.


Delete files uploaded to s3 from disk

This will allow us to shrink the asset NFS share with confidence, as the disk space used by assets will no longer grow continuously.

Trello card

This is actually unnecessary, as it happens already!
https://github.com/alphagov/asset-manager/blob/3d1496932b06f7ee965f3e7ba2124b8dccc8a37b/app/models/asset.rb#L80-L83
However, 12GB of empty directories have built up.


Load balancer terminated SSL documentation/configuration?

[yes] I read and understood how to enable logging

Issue / Steps to reproduce the problem
I'm having an issue getting IdentityServer4 to function correctly (ideally using the IdentityModel DiscoveryClient) when IDS is behind a load balancer.
The setup is there is a load balancer which is setup to accept HTTPS SSL only. This proxies the request to one of two servers running IDSv4 behind IIS. The load balancer to server link is HTTP.
In the version 3 documentation there were functions mentioned that should support this scenario on this page: https://identityserver.github.io/Documentation/docsv2/advanced/deployment.html
The version 4 documentation is not complete for the same topic: https://identityserver4.readthedocs.io/en/release/topics/deployment.html
In the discovery document all the endpoint URLs are returned as HTTP. I believe I need them to return as HTTPS and IdentityServer to understand that it's real public address is HTTPS.
I've tried some of the options under AddIdentityServer in Startup, like the IssuerUrl or relaxing the client restriction (not ideal), but I'm bouncing between the endpoint URLs being wrong or errors like issuer not matching authority from the client http vs https.
Relevant parts of the log file
I don't think I need to attach a log file at this point. I can certainly do it if there's value in it, let me know.

When you're using the UseIISIntegration extension (https://github.com/IdentityServer/IdentityServer4/blob/dev/src/Host/Program.cs#L27) it should set the incoming host header from the host of IIS. Are you not seeing that?

We are, but you run into the issue that IIS believes it's running on HTTP (it is), and so does IDS (since it gets it form IIS). But the requests come in to the load balancer as HTTPS, terminate there and then continue as HTTP to the servers.
This doesn't work without being able to tell IDS that it's external endpoint is really HTTPS, as seemes to be suggested in the linked v3 documentation.
We decided to just work around the problem for now with doing full SSL pass through so it's HTTPS throug the whole chain, but I do still want to know if that configuration/feature is available or coming.

Yes, the LB is in front of the IIS server. I'm going to close this issue. As far as I'm concerned it's far easier in our scenario to not off-load SSL and solve this problem that way.
I think writing a middleware to solve the issue that ID4 is listening on HTTP, but needs to believe it's listening on HTTPS and still have all the validation on issuer/etc pass is too high of a chance for issues and in this case would be another piece to maintain.
Thanks for the answers.

@sbobkin does the LB has the feature to pass a customised header to the the identity server?

@brockallen do you know which headers IdentityServer4 looking for?

host, mainly
check the microsoft middleware to see what they forward to know the full list.

When I connect over HTTPS to the load-balancer, Identity Server's "https://.../.well-known/openid-configuration shows http:// addresses.  I need to override this with https://.  Any thoughts?

@megamindbrian please find the following code, install package Microsoft.AspNetCore.HttpOverrides if you don't have.
            app.UseForwardedHeaders(new ForwardedHeadersOptions             {                 ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto,                 // the first local ip address for X-Forwarded-* from local                 // the second ip address mainly for docker default network, it may not apply based on the docker settings.                 // match the all ips, at the moment, it hard to know which local network it setups                 KnownNetworks =                 {                     // new IPNetwork(IPAddress.Loopback, 8),                     // new IPNetwork(IPAddress.Parse("<IP_ADDRESS>"), 8)                     new IPNetwork(IPAddress.Any, 0)                 }             });


Attributes on Dragging Components

When I dragging the mj-text component, their attributes are removing on drag to canvas.

data-gjs-editable="false" <= (this will remove on drag to canvas).
How should I fix this issue? I need the custom attributes while dragging but it removes.

This should be fixed in the latest release.


snyk: minimize the exclude list

rather than add blanket excludes, we should minimize the exclude list and target individual files whenever possible

(partially) solved by #157.
The next step should be adding reasons for exclusion, but snyk syntax is surprisingly clunky.


Clear cached udp connections on resolver Stop()

If the cached connections are not cleared sending the query to the corresponding external server fails for containers with the restart policy configured.
Signed-off-by: Santhosh Manohar<EMAIL_ADDRESS>

LGTM


Server starts on port 5000 but links in emails all point to port 3000

When the server starts (via bin/setup or foreman start,) it starts on port 5000 but the links in the confirmation email that come up refer to port 3000 and so don't work.

In http://stackoverflow.com/a/9804384/917187 the author claims that the -p flag is ignored within the Procfile. I can't find any evidence that this is true however, but in my searching I worked out your command syntax is wrong in the Procfile. I have raised a general question on SO about the correct way to specify a default port in a Procfile as I couldn't actually find anything.

The correct port is used if you run foreman start -p 3000

Please edit your config/config.yml file.
Settings you are looking for:
    mailer_url_options:
      host: localhost
      port: 3000


 shouldn't be used for underline

Hi!
I've found out that when applying underline, text gets wrapped in tag <ins>. That is wrong semantically. <ins> tag should be used to annotate new, added parts to document, not to be used as underline. To do it properly, you should use <span style="text-decoration: underline;"> (using <u> is not an alternative, since it also have different semantics, despite being used as underline).
I think it would be good to fix that, so documents are not only looking good, but are also semantically correct.
References:

Usage of <ins> tag: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/ins
Usage of <u> tag: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/u

Cheers!

To give you another argument for not using , I've created this HTML:
`

normal text
underlined text by ins
underlined text by u
underlined text by span

`
and I've opened it in Word and got this:
![image](https://user-images.githubusercontent.com/9076255/77892114-5704ec00-7272-11ea-987a-edaa50264efb.png)
The vertical line on the left is a Word annotation, that this paragraph has been added to the original document.
That's somehow similar to my use case of Suneditor. Results of what has been entered by user I'm sending as HTML to the backend and on backend I'm creating Word files out of these HTMLs. Unfortunately, whatever way I'm doing the parsing (external libraries or inserting HTML as AltChunk to docx),  is always treated as something inserted to the file, not underline. For now, I'm using a workaround, by replacing all usages of  with , but it would be better if editor wouldn't create wrong semantically output.

Hi @tswistak
sorry for late reply.😔
When editing the underline, I was mistaken for the <ins> as a substitute for the <u>.
I think it's correct to switch to the <u>.
I have seen so many different editors.
All editors using the <u> without exception.
Buttons such as "bold", "underline", "italic".. are buttons to insert different tags.
If to accurately reflect the meaning of the HTML tag, it will need to replace all buttons with a <span>.
I think most of the users will use the underline button expecting the <u> to be inserted.

I think that <u> can be used. Maybe it won't be exactly "just underline" but every viewer and editor will treat it just as underline.
Version with <span> could be a future improvement, if you'd like to switch from tags like <b> to inline styles in <span> like e.g. DraftJS does.

OK, I'll change  "ins" to "u" in next version.
For now, I will follow the way other editors.
The tag issue seems to have to look for more information and consider it.
After modification, it is replaced as follows.
<ins> -> <u>

The 2.28.1 version has been updated.
If this issue has not been resolved, please reopen this issue.
Thank you.


Authentication problem - client-id & key

I don't have an app created in my account and I want to use the java API to CRUD resource groups, VMs, etc.  What is the client-id for my username? or what should I put on the credentials auth? UserTokenCredentials asks for client-id, domain, username, password and azureenvironment. If my username is<EMAIL_ADDRESS>how can I create the credential?
I have also tried service principal by using az ad sp create-for-rbac but still I don't know what is the client-id and the key if I don't have an app created.
Thanks!

Hi @lukkamor
az ad sp create-for-rbac gives you an AppId - that's the client-id you are looking for.

so where I can find the key?

The key is also called the secret sometimes. I assume the python script worked for you.


Second part of #13846 that was missed (Add queue name to HttpSysOptions) - add option to connect to existing Q

Original request for 'Add queue name to HttpSysOptions #13846' was checked-in but missed the second part which was to support the ability to connect to an existing Q.  This is to add the missing behavior.  Code is available and branch pushed.

I have a fix for this

Isn't this is a duplicate of https://github.com/aspnet/AspNetCore/issues/5866?

Actually it was intended to be a part of #13846 that got missed.
I have no problem in how you want to deliver it, just that we need it.
It sounded on the other thread you did not want to deliver it so I need to look for a workaround.
However, if you want to do it that is great.
We can actually configure in our own management process if that helps.
From: Chris Ross<EMAIL_ADDRESS>Sent: Thursday, September 19, 2019 10:47 PM
To: aspnet/AspNetCore<EMAIL_ADDRESS>Cc: Brian Kress<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [aspnet/AspNetCore] Second part of #13846 that was missed (Add queue name to HttpSysOptions) - add option to connect to existing Q (#14138)
Isn't this is a duplicate of #5866https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Faspnet%2FAspNetCore%2Fissues%2F5866&data=02|01|Brian.Kress%40microsoft.com|1b41980913594ddbf0b608d73d103ce9|72f988bf86f141af91ab2d7cd011db47|1|0|637045012264884888&sdata=fc0REubZE9B135lKOuYSi%2BcH2FOpzpc8tSwPhJGK5HU%3D&reserved=0?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Faspnet%2FAspNetCore%2Fissues%2F14138%3Femail_source%3Dnotifications%26email_token%3DAKV7FBBEXUQDV2TD3WISUO3QKOGGRA5CNFSM4IYIGXH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7DXE2A%23issuecomment-533164648&data=02|01|Brian.Kress%40microsoft.com|1b41980913594ddbf0b608d73d103ce9|72f988bf86f141af91ab2d7cd011db47|1|0|637045012264884888&sdata=TcXQEFNaKTYjukMuUr4rsZuW2q3foNpb09dA6p5Xlqo%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKV7FBHCTPWYXWY4P722EA3QKOGGRANCNFSM4IYIGXHQ&data=02|01|Brian.Kress%40microsoft.com|1b41980913594ddbf0b608d73d103ce9|72f988bf86f141af91ab2d7cd011db47|1|0|637045012264894882&sdata=16TxXjb976hx%2FOAw4zM%2F9tDNAyKRJJUlQWwm6T1%2Bc%2Bw%3D&reserved=0.

Closing as a duplicate. We can re-evaluate #5866 to see if it's viable for 3.1. The biggest reason we did not pursue it in the past was lack of customer demand 😁


add 'sheeter' cloud function to project

sheeter is an api interface to google sheets
make a GET request to sheeter to retrieve results form sheets
make a PUT request to submit results to sheets
see example of sheeter in action in utils/dbApi.js
add workflows to run tests for sheeter and deploy sheeter

Notes for the reviewer
I haven't yet set up a test environment with shared creds that people can use to test the cloud function yet..
if you'd like to try running the cloud function locally, some setup is required:

you need to do a bunch of setup in GCP:

create a google cloud project in GCP  (follow instructions in the 'Before you begin' section of this tutorial)
enable the google sheets API
create a google sheet for cloud function to access (you will need to update cloud function to use your google sheet!)
create a google service account
create creds that can be used by locally run cloud function to access google
sheets

create a fork of the project (or clone the project)
go into the cloud-functions/ directory, and run following commands:

cd cloud-functions

# this project uses `nvm` to configure node environment
# highly recommend using it!
# otherwise, try using node v14.17.3 with npm v7.20.0
nvm use

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/creds.json"

## run cloud function
node node_modules/@google-cloud/functions-framework --target=sheeter

use curl to test cloud function (cloud function should be listening on http
://localhost:8080

## GET all results from google sheets
curl http://localhost:8080

## send something to google sheets
curl -X POST -H 'Content-Type: application/json' http://localhost:8080 -d '["hello"]'

N.B: These are very rough instructions, and may take some trial and error to get a test environment up to run cloud function locally! Please reach out to @hqtan if you need some help setting up!

Closing this PR. It's looking likely that we will deploy sheeter api as part of app to Vercel, which will really simplify deployment. With sheeter deployment to cloud function, we'd have to maintain a separate workflow that deploys the api separately. While it will mostly be automated, it's still a moving part that we'd have to look after.
The Vercel spike has been quite promising, and showed that we could bundle the deployment of the sheeter api as part of app deployment, simplifying the deployment process.


track mongodb _id field so we can attempt to reissue queries

[ ] CHANGELOG.md updated (feel free to wait until changes have been reviewed by a maintainer)

This change brings back the ability to re-issue a query after an err has been encountered while iterating a collection. For now, we will only attempt to perform the retry if the _id field is properly indexed is one of bson.ObjectId, string, float64, time.Time.

@trinchan good point on incremental addition of tests for the helper function. I plan on getting another PR in soon to lay a foundation for good e2e tests for adaptors.


Flutter with native iOS FCM plugin does not return from native code on initial run

Steps to Reproduce

A StackOverflow post has been posted here explaining the problem including code snippets and logs when running the application.
TLDR: I need to use the native FCM plugin for Android because I need to design custom vibrations when a notification is delivered. Then I want to use the native iOS plugin as well as I find it silly to implement the native Android plugin and the Flutter FCM package only to use it for the iOS platform. However, when I launch the app for the first time (clean install) on iOS, the token is not delivered. Every time I run it afterwards the token is delivered as expected.
Logs

Run
Due to the log exceeding the maximum character limit I have created a gist for the log (link).

Analyze
Analyzing [APP]...

warning • The parameter 'home' is required • lib/main.dart:48:48 • missing_required_param
warning • The parameter 'onSignedOut' is required • lib/main.dart:48:48 • missing_required_param
warning • The parameter 'webSocketChannel' is required • lib/main.dart:50:38 • missing_required_param
warning • The parameter 'sensorId' is required • lib/main.dart:53:42 • missing_required_param
warning • The parameter 'webSocketChannel' is required • lib/main.dart:53:42 • missing_required_param
warning • The parameter 'onSignedOut' is required • lib/main.dart:55:41 • missing_required_param
warning • The parameter 'documentID' is required • lib/main.dart:58:43 • missing_required_param
warning • The parameter 'documentID' is required • lib/main.dart:59:48 • missing_required_param
warning • The parameter 'name' is required • lib/main.dart:60:45 • missing_required_param
warning • The parameter 'sensorId' is required • lib/main.dart:60:45 • missing_required_param
warning • The parameter 'webSocketChannel' is required • lib/main.dart:60:45 • missing_required_param
   info • Cancel instances of dart.async.StreamSubscription • lib/pages/calibrate_sensor.dart:31:22 • cancel_subscriptions
   info • This class (or a class which this class inherits from) is marked as '@immutable', but one or more of its instance fields are not final: ClaimHubPage.fields, ClaimHubPage.onSubmit, ClaimHubPage.fieldWidth, ClaimHubPage.fontSize, ClaimHubPage.isTextObscure, ClaimHubPage.showFieldsAsBox • lib/pages/claim_hub.dart:5:7 • must_be_immutable
   info • The class 'Future' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions • lib/pages/claim_hub.dart:102:21 • sdk_version_async_exported_from_core
   info • This class (or a class which this class inherits from) is marked as '@immutable', but one or more of its instance fields are not final: PendingHome.fourDigitCode, PendingHome.crudMethods, PendingHome.auth • lib/pages/create_join_home.dart:8:7 • must_be_immutable
   info • The class 'Stream' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions • lib/pages/home_info.dart:15:3 • sdk_version_async_exported_from_core
   info • The class 'Stream' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions • lib/pages/join_household.dart:21:3 • sdk_version_async_exported_from_core
   info • The class 'Future' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions • lib/pages/join_household_view.dart:122:3 • sdk_version_async_exported_from_core
   info • The value of the field '_authStatus' isn't used • lib/pages/profile.dart:25:14 • unused_field
   info • The value of the field '_homeStatus' isn't used • lib/pages/profile.dart:26:14 • unused_field
   info • Name non-constant identifiers using lowerCamelCase • lib/pages/root_page.dart:154:8 • non_constant_identifier_names
   info • The class 'Future' was not exported from 'dart:core' until version 2.1, but this code is required to be able to run on earlier versions • lib/pages/sensor_dashboard_View.dart:131:3 • sdk_version_async_exported_from_core
   info • Unused import: 'package:APP/theme.dart' • lib/pages/source_location_view.dart:1:8 • unused_import
   info • This function has a return type of 'Future<String>', but doesn't end with a return statement • lib/util/crud_properties.dart:494:3 • missing_return
   info • This function has a return type of 'Future<String>', but doesn't end with a return statement • lib/util/crud_properties.dart:512:3 • missing_return
   info • Name non-constant identifiers using lowerCamelCase • lib/util/crud_properties.dart:667:16 • non_constant_identifier_names
   info • Unused import: 'dart:async' • lib/util/firebase_handler.dart:1:8 • unused_import
   info • Unused import: 'package:firebase_core/firebase_core.dart' • lib/util/firebase_handler.dart:2:8 • unused_import
   info • Unused import: 'package:cloud_firestore/cloud_firestore.dart' • lib/util/firebase_handler.dart:3:8 • unused_import
   info • The value of the field '_auth' isn't used • lib/util/firebase_handler.dart:28:23 • unused_field
   info • Unused import: 'package:APP/theme.dart' • lib/widgets/toggle_mute_icon.dart:3:8 • unused_import
   info • This class (or a class which this class inherits from) is marked as '@immutable', but one or more of its instance fields are not final: ToggleMuteIcon.unmuted • lib/widgets/toggle_mute_icon.dart:5:7 • must_be_immutable

Doctor
[✓] Flutter (Channel beta, v1.3.8, on Mac OS X 10.13.6 17G65, locale en-DK)
    • Flutter version 1.3.8 at /Users/markflarup/Library/flutter
    • Framework revision e5b1ed7a7f (3 weeks ago), 2019-03-06 14:23:37 -0800
    • Engine revision f4951df193
    • Dart version 2.2.1 (build 2.2.1-dev.0.0 571ea80e11)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/markflarup/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.1, Build version 10B61
    • ios-deploy 1.9.4
    • CocoaPods version 1.6.1

[✓] Android Studio (version 3.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 33.4.1
    • Dart plugin version 182.5215
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[✓] IntelliJ IDEA Ultimate Edition (version 2018.3.5)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 33.4.2
    • Dart plugin version 183.5912.23

[✓] Connected device (1 available)
    • iPhone7 • 7040ee04b430aab1fca1c0d9e32a93f48ed89d3b • ios • iOS 12.1.2

• No issues found!

Hi, sorry for the delay, I just want to check if you could solve this problem.

Hi, sorry for the delay, I just want to check if you could solve this problem.

No, I wasn't able to solve it. I had tried various options before posting here and haven't really tried any other solutions afterwards.
The project I worked on got shut down so it's not really an issue for me to tackle now.

Thanks for getting back, I close this issue then.


Flat Routes - Route Segment after Param

Hi Michael,
remix-flat-routes looks great.
I was trying to implement flat-routes on our current 'workbench' Remix app - starting with 'flat-files', before trying 'flat-folders' or 'nested' or 'hybrid'.
For the following 'flat-files' route files....
.
├── _auth.sign-in.tsx
├── _auth.sign-out.tsx
├── _auth.sign-up.tsx
├── _auth.tsx
├── actions.set-theme.tsx
├── app.notes.$noteId.delete.tsx
├── app.notes.$noteId.edit.tsx
├── app.notes.$noteId.tsx
├── app.notes.index.tsx
├── app.notes.new.tsx
├── app.tsx
├── healthcheck.tsx
├── index.css
└── index.tsx

Should the app.notes.$noteId.edit.tsx and app.notes.$notedId.delete.tsx routes work?
The server routes for  /app/notes/<note-id>/edit /app/notes/<note-id>/delete seem to be working  (the correct loader is being called), but in the browser, the edit and delete routes are re-rendering the /app/notes/<note-id> route and not the edit or delete React page components.
Are we doing something wrong?

I've just tried to reproduce this on a clean Indie Stack install and I'm pretty sure I'm seeing the same problem here. The server received the route, loader runs, but the browser router stays on the note details page - double rendering the components...
I've setup a test repo here - https://github.com/infonomic/flat-routes-test  -  again, based on a clean Indie stack install with the following routes.
.
├── _auth.join.tsx
├── _auth.login.tsx
├── _auth.logout.tsx
├── app.notes.$noteId.edit.tsx
├── app.notes.$noteId.tsx
├── app.notes.index.tsx
├── app.notes.new.tsx
├── app.notes.tsx
├── healthcheck.tsx
└── index.tsx

Hope some of this helps, and apologies in advance if I've somehow mixed things up.

Can you run npx remix routes and paste the output? Thanks

Does $noteId have an <Outlet>?

Sure - here's the results of npm remix routes from the test repo....
<Routes>
  <Route file="root.tsx">
    <Route path="join" file="routes/_auth.join.tsx" />
    <Route path="login" file="routes/_auth.login.tsx" />
    <Route path="logout" file="routes/_auth.logout.tsx" />
    <Route path="app/notes" file="routes/app.notes.tsx">
      <Route path=":noteId" file="routes/app.notes.$noteId.tsx">
        <Route path="edit" file="routes/app.notes.$noteId.edit.tsx" />
      </Route>
      <Route path="new" file="routes/app.notes.new.tsx" />
      <Route index file="routes/app.notes.index.tsx" />
    </Route>
    <Route path="healthcheck" file="routes/healthcheck.tsx" />
    <Route index file="routes/index.tsx" />
  </Route>
</Routes>

And no - neither app.notes.$noteId.tsx nor app.notes.$noteId.edit.tsx has an outlet (only app.notes.tsx does)

Ok that's what I thought. By default it is looking for the matching parent route.
Since $noteId matches on your new and edit routes, it thinks that's the parent.
Rename your routes to add the _ suffix to say it doesn't have an outlet. Then it will match the notes route
notes.$noteId_.new.tsx

Okay that worked.
<Routes>
  <Route file="root.tsx">
    <Route path="join" file="routes/_auth.join.tsx" />
    <Route path="login" file="routes/_auth.login.tsx" />
    <Route path="logout" file="routes/_auth.logout.tsx" />
    <Route path="app/notes" file="routes/app.notes.tsx">
      <Route path=":noteId" file="routes/app.notes.$noteId.tsx" />
      <Route path=":noteId/edit" file="routes/app.notes.$noteId_.edit.tsx" />
      <Route path="new" file="routes/app.notes.new.tsx" />
      <Route index file="routes/app.notes.index.tsx" />
    </Route>
    <Route path="healthcheck" file="routes/healthcheck.tsx" />
    <Route index file="routes/index.tsx" />
  </Route>
</Routes>

I do recall reading about the '_' suffix and it's behavior, but did put it together for this case. I'll go back and take a look at the docs again, and if you think some extra text or an example would help I'd be glad to submit a PR.

Yeah I need to write more documentation. I plan on doing some videos showing the different conventions as well.

Okay - we'll I'd be glad to lend a hand with the docs if I can be of help. Will close this issue if that's okay with you.

This is sweet. You guys are on another level ;-)
.
├── _actions+
│   └── actions.set-theme.tsx
├── _app+
│   ├── _app.tsx
│   ├── account.$userId.email.tsx
│   ├── account.$userId.password.tsx
│   ├── account.index.tsx
│   ├── notes.$noteId.tsx
│   ├── notes.$noteId_.delete.tsx
│   ├── notes.$noteId_.edit.tsx
│   ├── notes.index.tsx
│   └── notes.new.tsx
├── _auth+
│   ├── _auth.tsx
│   ├── sign-in.tsx
│   ├── sign-out.tsx
│   └── sign-up.tsx
├── _landing+
│   ├── _landing.tsx
│   ├── index.css
│   └── index.tsx
├── admin+
│   ├── _admin.tsx
│   ├── index.tsx
│   ├── users.tsx
│   └── users_.$userId.tsx
├── healthcheck.tsx
├── theme-radix-dialog.tsx
├── theme-radix-toast.tsx
├── theme-tailwind-forms.tsx
└── theme.tsx

Yeah, I really like the new + convention. It allows you to organize the routes and shorten the names.


Docs: deployment

Status: Ready for review!
I've changed a little bit of the base url stuff in Docusaurus, but all appears to work in localdev and staging deploys. You can see current staging at: https://formidable-com-react-native-owl-staging-68.surge.sh/open-source/react-native-owl/
I've hooked up what should be all the correct work for our production deploy to AWS at https://formidable.com/open-source/react-native-owl on merging to main, but we'll just want to watch that as we go.
@formidableamy note that we have a weird chicken-and-egg thing where we need to have the deploy of this project site right before a deploy of the related formidable.com changes.
Work:

[x] Update base url support.
[x] Add formideploy and basic workflow + docs
[x] Get formideploy working on staging
[x] Get formideploy working on production

[x] Add AWS token.

[x] Hone CI/CD to just docs/* + website/* changes

@manosim I'm getting a library/demo app visual regression failure that I'm guessing is completely unrelated to my docs changes after merging in latest main?
  App.tsx
  console.info
    ✕ takes a screenshot of the first screen (24652 ms)
    [OWL] Screenshot saved to /Users/runner/work/react-native-owl/react-native-owl/example/.owl/latest/ios/homescreen.png.

      at Logger.info (logger.js:25:26)

  ● App.tsx › takes a screenshot of the first screen

    Compared screenshot to match baseline. 6913 were different.

       5 |     const screen = await takeScreenshot('homescreen');
       6 |
    >  7 |     expect(screen).toMatchBaseline();
         |                    ^
       8 |   });
[OWL] Generating Report
       9 | });
      10 |


TaskEither from predicate should accept a FutureOr.

I would like to use the TaskEither from predicate with a async predicate.
I think this would fit into the TaskEither constructor very well.
The current implementation is this.
factory TaskEither.fromPredicate(
          R value, bool Function(R a) predicate, L Function(R a) onFalse) =>
      TaskEither(
          () async => predicate(value) ? Right(value) : Left(onFalse(value)));

It could be changed like so:
factory TaskEither.fromPredicate(
  R value,
  FutureOr<bool> Function(R a) predicate,
  L Function(R a) onFalse
) => TaskEither(
  () async {
    final result = await predicate(value);
    return result ? Right(value) : Left(onFalse(value));
  }
);

Would this be a good idea? If so I can submit a PR.

Hi @zellidev0
TaskEither.fromPredicate is intended for sync conditions, so I would keep the current implementation. For async conditions I would suggest first performing the async operation, and then chaining a function to check the result.
Let me know if you had some specific example where this would not work


Manual for disabling test coverage checking

What
Documentation for disabling test coverage by 2nd line.
Why
As explained in the comment in this Trello ticket

I'm closing this pull request based on the comment above. We already have a workaround for issues like that: https://docs.publishing.service.gov.uk/manual/deployments.html#manual-deployments


fix(cli): amplify plugin scan command print correct version of inactive hosting plugins

Description of changes

Fix missing plugin version case of exists multiple plugins in same category
Issue #, if available

fix #8127
Description of how you validated changes

Unit Testing (lerna --scope @aws-amplify/cli run test)
Manual Testing

Checklist

[x] PR description included
[x] yarn test passes
[x] Tests are changed or added
[x] Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #8130 (4a0b910) into master (b524870) will increase coverage by 0.16%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #8130      +/-   ##
==========================================
+ Coverage   54.43%   54.59%   +0.16%
==========================================
  Files         664      664
  Lines       37838    37842       +4
  Branches     7768     7769       +1
==========================================
+ Hits        20597    20660      +63
+ Misses      16401    16342      -59
  Partials      840      840

Impacted Files
Coverage Δ

...fy-cli/src/plugin-helpers/platform-health-check.ts
100.00% <100.00%> (+89.39%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b524870...4a0b910. Read the comment docs.

Hi @fossamagna, thank you for your contribution!