Submit Phase 1 data

Unfortunately we need a windows machine to QA the final spreadsheet, package and submit (from all I can tell anyway).
I have machine so Lucy if you are able to get the spreadsheet ready and photos to ~/Library/CloudStorage/OneDrive-Personal/Projects/submissions/PSCIS/2023/fraser I can take it from there
I will link to here with csv export from the two projects and script used to generate it.

just changed name of export submission (as above) to match purpose of last linked script.  Here is the latest version of the script to get all the photos into one place https://github.com/NewGraphEnvironment/fish_passage_skeena_2023_reporting/blob/main/scripts/01_prep_inputs/0140-pscis-export-submission.R
of course will need to be run twice to get photos from both QGIS projects vs just the one as in the Skeena

Actually - here is the last time an export was done.  it was simplified with fs calls and comes from just the reassessments section of fish_passage_moti_2022_reporting
## Reassessments-----------------------------------------------------------------
# this was updated July 2024 using Skeena 2023 as template
library(fpr)
library(tidyverse)
library(fs)

# PSCIS Submissions -------------

tfpr_filter_list <- function(idx){
  filestocopy_list[idx]
}

tfpr_photo_change_name <- function(filenames_to_change = filestocopy_list){
  gsub(filenames_to_change, pattern = path, replacement = targetdir)
}

name_repo <- 'fish_passage_moti_2022_reporting'
name_pdf <- 'Moti2022.pdf'
url_github <- 'https://github.com/NewGraphEnvironment/'
url_gitpages <- 'https://newgraphenvironment.github.io/'
name_submission <- 'pscis_reassessments.xlsm'

# need to add photos to local machine to upload to PSCIS
targetdir = fs::path('/Users/airvine/Library/CloudStorage/OneDrive-Personal/Projects/PSCIS/2022/reassessments', name_repo)

d <- fpr::fpr_import_pscis(workbook_name = 'pscis_reassessments.xlsm')

folderstocopy<- d$pscis_crossing_id %>% as.character()

path <- fs::path_wd('data/photos/')

path_to_photos <- fs::path(path, folderstocopy)

# here we transfer just the photos with labels over into the PSCIS directory where we will upload from to the gov interface

fs::dir_create(targetdir)

folderstocreate<- fs::path(targetdir, folderstocopy)

##create the folders
fs::dir_create(folderstocreate)

filestocopy_list <- path_to_photos %>%
  purrr::map(fpr::fpr_photo_paths_to_copy) %>%
  purrr::set_names(basename(folderstocreate))

##view which files do not have any photos to paste by reviewing the empty_files object
empty_idx <- which(!lengths(filestocopy_list))
empty_files <- empty_idx %>% tfpr_filter_list()

##rename long names if necessary

photo_sort_tracking <- path_to_photos %>%
  purrr::map(fpr::fpr_photo_document_all) %>%
  purrr::set_names(folderstocopy) %>%
  bind_rows(.id = 'folder') %>%
  mutate(photo_name = str_squish(str_extract(value, "[^/]*$")),
         photo_name_length = stringr::str_length(photo_name))

###here we back up a csv that gives us the new location and name of the original JPG photos.
## Not ideal because we did some sorting by hand without adding name of camera to the file name but a start on reproducibility nonetheless

##burn to csv
photo_sort_tracking %>%
  readr::write_csv(file = 'data/photos/photo_sort_tracking_reassessments.csv')

filestopaste_list <- filestocopy_list %>%
  map(tfpr_photo_change_name)

##!!!!!!!!!!!!!!!copy over the photos!!!!!!!!!!!!!!!!!!!!!!!
mapply(fs::file_copy,
       path =  filestocopy_list,
       new_path = filestopaste_list)

##also move over the pscis file
fs::file_copy(path = fs::path('data', name_submission),
              new_path = fs::path(targetdir, name_submission),
              overwrite = T)

#make a little readme for the pdf for upload to ecocat and other details
writeLines(
  paste(
    "Online interactive report is located at: ",
    paste0(url_gitpages, name_repo),
    "",
    "A versioned pdf of the report can be downloaded from: ",
    paste0(url_github, name_repo, "/raw/main/docs/", name_pdf),
    "",
    "Raw data is available here: ",
    paste0(url_github, name_repo, "/tree/main/data"),
    "",
    "All scripts to produce online interactive reporting and pdf are located at: ",
    paste0(url_github, name_repo),
    sep = "\n"
  ),
  fs::path(targetdir, 'readme.txt')
)

##in the future we will need to add the copy calls to move the directory off of onedrive to
#  the windows machine used for project submission.  Don't want to set up the machine right now
# so will do it by hand. What an insane pain

cool. I'll get on it

csvs have time attached to the date, we will have to fix and reburn to csv.

nevermind! found the date column! dont use date_time_start, use date

submission uploaded to ESF as submitssion id: 2369231
wow.
classic PSCIS submission absolute time suck insanity.
here is a screenshot of the xml file errors overcome. the errors related to the crossing subtype make sense (although the excel validation missed them) as I had accidentally erased "Ford" from the submission spreadsheet before copy pasting from .csv to the .xlsx template.  fixed these before rezip and new xml file generation
The first error on ESF upload though makes no sense. For some reason 50 from the Percent Backwatered' field of crossing 13903618did not transfer from thexlsxfile to the.xmlduring theGenerate Submission` phase of the upload.  Not sure at all why. To overcome needed to:

use error message info to track down where the problem was (ie. row 6356 of the xml file).
use proper text editor (sublime text - maybe rstudio could have worked too) to edit the xml file to include 50 in the xml file
resubmit with the hand bombed xml file

Even though it is "accepted" it may not land.  We have had many many many long and painfull battles with this system over the years trying to get data to land so we may very well recieve cryptic error messages tomorrow morning that it didn't land. fingers crossed though. will close issue if it does land

ok so here are the errors I get. I couldn't find the screenshots I took on the PC so sorry for the poor quality.
Last 4 I fixed  with adding in ford as per above.
Looks like the backwater issue is still there as well. I will fix that.
I'll look into the other issues tomorrow and get it submitted tomorrow. Did my best today but just ran out of time.

PSCIS data has been submitted.
Above issues were fixed by:

Adding in Funding source as OTHER
Adding in the budget amount, i used 30,000 which is what it said in the 2023 skeena phase 1 spreadsheet.
As per above, Added in Ford.
As per above, Edited the XML file in Sublime text and changed backwatered to 50. See first screenshot.

awesome. Thanks!
If we need to do it again (maybe likely - but hopefully not) we can put MoTi for the funder.   Think the submissions were made from the Documents/submissions (or something like that) directory on the PC so I think the old submission should be there for reference
It is incredibly odd that the backwater error reappears after a fresh copy and paste to the template. Makes me think it must be the template causing the issue...  Can point them to this issue eventually once we hopefully get the data to land....

20241028 Submission got rejected. Below are the errors:
2 errors unrelated to photos are documented here https://github.com/NewGraphEnvironment/fish_passage_fraser_2023_reporting/issues/113#issuecomment-2445358469 and https://github.com/NewGraphEnvironment/fish_passage_fraser_2023_reporting/issues/113#issuecomment-2445366318. ChatGPT was helpful for finding these amid all the text, https://chatgpt.com/share/e/67217347-a180-8012-afb8-cb6cd3c069f0
Crossing: 13903627 Attachment: TimePhoto_20231004_093006_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903451 Attachment: TimePhoto_20231003_132949_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200051 Attachment: TimePhoto_20231005_155342_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900308 Attachment: TimePhoto_20231003_125007_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903148 Attachment: TimePhoto_20231005_094718_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 5400019. The supplied value of 99.9 is not valid for Structure Length. The value supplied must be >0 and <=99.0.
Crossing: 13900003 Attachment: TimePhoto_20231005_110423_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900015 Attachment: TimePhoto_20231005_095647_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900025 Attachment: TimePhoto_20231005_085616_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_122353_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903148 Attachment: TimePhoto_20231005_094047_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900201 Attachment: TimePhoto_20231004_142633_road.JPG. The process was unable to resize th More Info
2024-10-28 10:34:32
e attachment photos down to the required size of 1024X1024.
Crossing: 9900446 Attachment: TimePhoto_20230930_152258_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900003 Attachment: TimePhoto_20231005_111108_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900193 Attachment: TimePhoto_20231004_112913_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900265 Attachment: TimePhoto_20231004_115112_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900309 Attachment: TimePhoto_20231003_120850_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900015 Attachment: TimePhoto_20231005_095712_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_123347_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_122822_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900050 Attachment: TimePhoto_20231003_163405_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900064 Attachment: TimePhoto_20231004_172525_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900030 Attachment: TimePhoto_20231005_114229_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
More Info
2024-10-28 10:34:33

Crossing: 13900252 Attachment: TimePhoto_20231003_144816_o_utlet2.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900064 Attachment: TimePhoto_20231004_171418_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900019 Attachment: TimePhoto_20231004_155945_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900012 Attachment: TimePhoto_20231005_125512_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900012 Attachment: TimePhoto_20231005_124740_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900201 Attachment: TimePhoto_20231004_143718_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900003 Attachment: TimePhoto_20231005_111332_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900043 Attachment: TimePhoto_20231003_173308_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900201 Attachment: TimePhoto_20231004_142957_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22201229 Attachment: TimePhoto_20231005_142818_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900077 Attachment: TimePhoto_20231005_104401_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900309 Attachment: TimePhoto_20231003_1 More Info
2024-10-28 10:34:33
21000_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 9900446 Attachment: TimePhoto_20230930_153226_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900193 Attachment: TimePhoto_20231004_111948_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900026 Attachment: TimePhoto_20231003_113805_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900077 Attachment: TimePhoto_20231005_104345_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_134123_o_utletembankment.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903184 Attachment: TimePhoto_20231004_100239_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903450 Attachment: TimePhoto_20231003_134731_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900026 Attachment: TimePhoto_20231003_113631_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903627 Attachment: TimePhoto_20231004_093501_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900309 Attachment: TimePhoto_20231003_121954_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_133512_upstream.JPG. The process was unable to res More Info
2024-10-28 10:34:33
ize the attachment photos down to the required size of 1024X1024.
Crossing: 13900198 Attachment: TimePhoto_20231004_132808_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900308 Attachment: TimePhoto_20231003_124706_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200029 Attachment: TimePhoto_20231005_151216_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903148 Attachment: TimePhoto_20231005_093630_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_123002_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200029 Attachment: TimePhoto_20231005_151655_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900019 Attachment: TimePhoto_20231004_161233_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200051 Attachment: TimePhoto_20231005_155234_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900193 Attachment: TimePhoto_20231004_112128_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900265 Attachment: TimePhoto_20231004_115325_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900052 Attachment: TimePhoto_20231003_152738_downstream.JPG. The process was unable to resize the attachment photos down to the require More Info
2024-10-28 10:34:33
d size of 1024X1024.
Crossing: 13905581 Attachment: TimePhoto_20231003_160503_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900052 Attachment: TimePhoto_20231003_153228_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903184 Attachment: TimePhoto_20231004_095801_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 24715750 Attachment: TimePhoto_20231003_110932_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900064 Attachment: TimePhoto_20231004_172533_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900050 Attachment: TimePhoto_20231003_163258_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900019 Attachment: TimePhoto_20231004_160538_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900015 Attachment: TimePhoto_20231005_100220_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900192 Attachment: TimePhoto_20231004_103945_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 24715750 Attachment: TimePhoto_20231003_111011_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903451 Attachment: TimePhoto_20231003_132650_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903184 A More Info
2024-10-28 10:34:33
ttachment: TimePhoto_20231004_100514_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900052 Attachment: TimePhoto_20231003_153312_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900265 Attachment: TimePhoto_20231004_120542_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200051 Attachment: TimePhoto_20231005_155200_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900077 Attachment: TimePhoto_20231005_105119_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900192 Attachment: TimePhoto_20231004_104050_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900043 Attachment: TimePhoto_20231003_171551_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900201 Attachment: TimePhoto_20231004_144839_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900043 Attachment: TimePhoto_20231003_171517_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900064 Attachment: TimePhoto_20231004_170347_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200051 Attachment: TimePhoto_20231005_155252_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900050 Attachment: TimePhoto_20231003_163624_barrel.JPG. The proces More Info
2024-10-28 10:34:33
s was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900052 Attachment: TimePhoto_20231003_152729_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900265 Attachment: TimePhoto_20231004_115604_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903627 Attachment: TimePhoto_20231004_093550_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903451 Attachment: TimePhoto_20231003_132642_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900265 Attachment: TimePhoto_20231004_120110_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900308 Attachment: TimePhoto_20231003_130626_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_122509_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900193 Attachment: TimePhoto_20231004_111507_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_134209_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900201 Attachment: TimePhoto_20231004_143157_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200029 Attachment: TimePhoto_20231005_151115_road.JPG. The process was unable to resize the attachment photos down to the required More Info
2024-10-28 10:34:33
size of 1024X1024.
Crossing: 13900025 Attachment: TimePhoto_20231005_090133_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900019 Attachment: TimePhoto_20231004_161840_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900025 Attachment: TimePhoto_20231005_085718_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900193 Attachment: TimePhoto_20231004_112612_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900193 Attachment: TimePhoto_20231004_112945_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903450 Attachment: TimePhoto_20231003_135306_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900077 Attachment: TimePhoto_20231005_104112_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_144808_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_124212_i_nletpic2.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900003 Attachment: TimePhoto_20231005_110348_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900192 Attachment: TimePhoto_20231004_104031_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 9900446 Attachm More Info
2024-10-28 10:34:33
ent: TimePhoto_20230930_153734_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_122523_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900030 Attachment: TimePhoto_20231005_114329_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22201229 Attachment: TimePhoto_20231005_143446_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903450 Attachment: TimePhoto_20231003_135805_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900043 Attachment: TimePhoto_20231003_172225_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900192 Attachment: TimePhoto_20231004_105825_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900019 Attachment: TimePhoto_20231004_160307_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13905581 Attachment: TimePhoto_20231003_160502_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900198 Attachment: TimePhoto_20231004_132901_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900200 Attachment: TimePhoto_20231004_135856_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900198 Attachment: TimePhoto_20231004_130535_road.JPG. The process was More Info
2024-10-28 10:34:33
unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_124249_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_122754_weirdownstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900012 Attachment: TimePhoto_20231005_125535_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900026 Attachment: TimePhoto_20231003_113230_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903184 Attachment: TimePhoto_20231004_095634_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900019 Attachment: TimePhoto_20231004_162503_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903148 Attachment: TimePhoto_20231005_093742_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900003 Attachment: TimePhoto_20231005_110210_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 9900446 Attachment: TimePhoto_20230930_152407_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_145421_i_nlet2.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_123318_downstream.JPG. The process was unable to resize the attachment photos down to More Info
2024-10-28 10:34:33
the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_133234_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22201229 Attachment: TimePhoto_20231005_143314_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903627 Attachment: TimePhoto_20231004_093019_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900012 Attachment: TimePhoto_20231005_131631_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900064 Attachment: TimePhoto_20231004_170928_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13905581 Attachment: TimePhoto_20231003_160553_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200051 Attachment: TimePhoto_20231005_155357_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900030 Attachment: TimePhoto_20231005_114351_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_134131_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900050 Attachment: TimePhoto_20231003_163454_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900026 Attachment: TimePhoto_20231003_113154_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900309 At More Info
2024-10-28 10:34:34
tachment: TimePhoto_20231003_121228_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22201229 Attachment: TimePhoto_20231005_142655_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900309 Attachment: TimePhoto_20231003_121923_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903450 Attachment: TimePhoto_20231003_134907_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900043 Attachment: TimePhoto_20231003_171719_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900200 Attachment: TimePhoto_20231004_140405_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900064 Attachment: TimePhoto_20231004_170950_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900030 Attachment: TimePhoto_20231005_114315_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900003 Attachment: TimePhoto_20231005_111158_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200029 Attachment: TimePhoto_20231005_151636_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22201229 Attachment: TimePhoto_20231005_143709_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200029 Attachment: TimePhoto_20231005_151200_outlet.JPG. More Info
2024-10-28 10:34:34
The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900026 Attachment: TimePhoto_20231003_113014_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900198 Attachment: TimePhoto_20231004_131518_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900200 Attachment: TimePhoto_20231004_140333_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900030 Attachment: TimePhoto_20231005_114433_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_122204_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_144758_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900050 Attachment: TimePhoto_20231003_163944_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_144306_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900309 Attachment: TimePhoto_20231003_122656_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900052 Attachment: TimePhoto_20231003_152526_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_124159_upstream.JPG. The process was unable to resize the attachment photos down More Info
2024-10-28 10:34:34
to the required size of 1024X1024.
Crossing: 13900201 Attachment: TimePhoto_20231004_145203_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900308 Attachment: TimePhoto_20231003_125528_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13905581 Attachment: TimePhoto_20231003_155842_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900192 Attachment: TimePhoto_20231004_105222_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903184 Attachment: TimePhoto_20231004_101327_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900025 Attachment: TimePhoto_20231005_085743_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900077 Attachment: TimePhoto_20231005_104332_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_133948_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903451 Attachment: TimePhoto_20231003_132353_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903627 Attachment: TimePhoto_20231004_093239_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900050 Attachment: TimePhoto_20231003_163613_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Cros More Info
2024-10-28 10:34:34
sing: 13900077 Attachment: TimePhoto_20231005_105114_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 9900446 Attachment: TimePhoto_20230930_151848_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903450 Attachment: TimePhoto_20231003_135825_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903450 Attachment: TimePhoto_20231003_134851_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903148 Attachment: TimePhoto_20231005_093052_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903451 Attachment: TimePhoto_20231003_132152_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903148 Attachment: TimePhoto_20231005_093342_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200022 Attachment: TimePhoto_20231005_133505_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 9900446 Attachment: TimePhoto_20230930_153602_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900200 Attachment: TimePhoto_20231004_135425_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900012 Attachment: TimePhoto_20231005_125244_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900015 Attachment: TimePhoto_20231005_095732_upstream.JPG. Th More Info
2024-10-28 10:34:34
e process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22200051 Attachment: TimePhoto_20231005_155239_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900198 Attachment: TimePhoto_20231004_131642_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 24715750 Attachment: TimePhoto_20231003_111005_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903627 Attachment: TimePhoto_20231004_092713_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900025 Attachment: TimePhoto_20231005_090938_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900012 Attachment: TimePhoto_20231005_124756_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 24715750 Attachment: TimePhoto_20231003_111031_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900030 Attachment: TimePhoto_20231005_114452_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_144421_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900052 Attachment: TimePhoto_20231003_152750_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900015 Attachment: TimePhoto_20231005_095541_road.JPG. The process was unable to resize the attachment photos dow More Info
2024-10-28 10:34:34
n to the required size of 1024X1024.
Crossing: 13900026 Attachment: TimePhoto_20231003_113706_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900025 Attachment: TimePhoto_20231005_090825_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900073 Attachment: TimePhoto_20231005_121947_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13905581 Attachment: TimePhoto_20231003_160616_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900308 Attachment: TimePhoto_20231003_124309_road.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903451 Attachment: TimePhoto_20231003_132258_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900192 Attachment: TimePhoto_20231004_104655_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_144408_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900015 Attachment: TimePhoto_20231005_100238_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900198 Attachment: TimePhoto_20231004_132736_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13903184 Attachment: TimePhoto_20231004_101131_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 139 More Info
2024-10-28 10:34:34
00043 Attachment: TimePhoto_20231003_173048_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 22201229 Attachment: TimePhoto_20231005_142633_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900270 Attachment: TimePhoto_20231004_123557_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900252 Attachment: TimePhoto_20231003_144442_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900265 Attachment: TimePhoto_20231004_115338_downstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900308 Attachment: TimePhoto_20231003_124424_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 24715750 Attachment: TimePhoto_20231003_110951_inlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 13900200 Attachment: TimePhoto_20231004_140317_outlet.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 24715750 Attachment: TimePhoto_20231003_110955_upstream.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.
Crossing: 5400024. The supplied value of 12.0 is not valid for Structure Diameter or Span. If stream crossing type is Closed Bottom Structure then the value supplied must be > 0 and <=9.99 meters. If stream crossing type is Open Bottom Structure then the value supplied must be > 0 and <=300 meters. If the crossing type is Other than the value is not required.
Crossing: 22200029 Attachment: TimePhoto_20231005_ More Info
2024-10-28 10:34:34
153139_barrel.JPG. The process was unable to resize the attachment photos down to the required size of 1024X1024.

interesting that there were 53 less errors... That was the result only the 1 additional photo that was renamed (by me) in the most recent submission vs the one before? Perhaps reply in issue if that assumption is incorrect
made fpr::fpr_photo_qa2 to test for dupes. non found.
logged metadata strip to reduce metadata columns of all photos (sometime over 100 to 20 or so) with following common warning logged (still works just issues warning):
"Warning: ICC_Profile deleted. Image colors may be affected"
Interestingly this occured exactly 223 times out of the 1113 photos processed. Doesn't quite match our number of errors on the last submission attempt but sort of close.
reprocessed photos are in
~/Library/CloudStorage/OneDrive-Personal/Projects/submissions/PSCIS/2023/fraser/photo_meta_rm
will point here with latest scripts to reproduce as well as log of strip details

I changed probably 5 more photos after you did, so there was definitely more than 1 photo renamed between yesterday’s submission and Friday.

20241031 Submission is submitted. Fixes include:

hacked XML to fix backwatering issue
photos stripped of meta data

Data has been successfully submitted.
Closing in favour of

metadata strip https://github.com/NewGraphEnvironment/fish_passage_guidebook/issues/13
reserve photo duplicates https://github.com/NewGraphEnvironment/fish_passage_guidebook/issues/12


Fix validation printing workers object on error

What this PR does / why we need it:
We have a validation rule ensuring that if machine-controller is disabled, there must not be workers defined in the manifest or Terraform state. The validation marks the workers field as invalid and prints the whole object on stdout/stderr. The output is too long and may contain sensitive data, so we should not print it like that.
Instead, this PR updates the validation rule to mark the field as forbidden in such cases, which prevents validation from printing the whole object.
Does this PR introduce a user-facing change?:
NONE

/assign @kron4eg

/retest

/retest

/retest


Fail to connect , state -4 MQTT_CONNECTION_TIMEOUT

I'm trying to run "Basic ESP8266 MQTT example" with nodemcu v3 and mosquitto broker set up on raspberry pi.
Unfortunately all the time I'm getting connection state -4
"Attempting MQTT connection...failed, rc=-4 try again in 5 seconds"
According to doc, its
MQTT_CONNECTION_TIMEOUT - the server didn't respond within the keep alive time
This is strange because I cant publish/subscribe using .net client with m2mqtt library.
On raspberry pi I can ping nodemcu module
Any ideas what should I check ?

I'm trying to connect Azure IoT Hub from ESP8266 via MQTT and I'm getting exactly same rc which is -4.
My proposal is something about SSL.
In addition,
I tried with cloudmqtt.com, it is working without ssl, but when I tried to connect with SSL port to cloudmqtt.com with exactly same credentials , I got rc -4,

I'm trying to connect to cloudmqtt.com without ssl, but got rc 5 (MQTT_CONNECT_UNAUTHORIZED)
WiFiClient espClient;
PubSubClient client("m21.cloudmqtt.com", 14069, 0, espClient);
client.connect("USERNAME1", username, pass))

Maybe you got any idea what might be the cause ?

I am using PubSubClient 2.6.0 and if I try connect in a MQTT Broker in local network, that works, if the Broker is in other network I got -4 error.
I can connect from my local network (using nat) in that Broker and my Arduino can ping the Broker IP.
What is happening?

@smashpl Exactly the same here. I'm trying to connect with NodeMCU v2 and v3 to mosquitto broker in a Raspberry Pi and I'm getting the same exact error.

When running MQTT broker mosquitto armhf 0.15-2 on Raspberry Pi  wheezy,
to avoid timeout errors "MQTT_CONNECTION_TIMEOUT" -4 when connecting,
try setting the version in PubSubClient.h to the older version 3_1
// MQTT_VERSION : Pick the version
#define MQTT_VERSION MQTT_VERSION_3_1 <<- uncomment this line
Mosquitto 0.15-2 apparently does not support the newer version 3_1_1.
This got it working for me.

Setting MQTT_VERSION to 3.1 fixed my error -4 also.
Thanks!

I also searched for hours. This should be improved:

get different error for version discrepancy
autoresolve protocol version by doing retry of call and store version for current session.

Hi anyone,
I'm very much afraid I'm having exactly the same issue (see my post above).
Even I'm fairly ok in doing some Linux via ssh, can anyone tell me the necessary steps to compile everything needed and install mosquitto once I changed the parameter?
Thanks.

If you are using multiple "Things", you have to create different Certificates for each IoT Device... Also, when you connect the client, you have to change the name:
 if (client.connect("CHANGE_THIS_IT_HAS_TO_BE_DIFFERENT_FOR_EACH_DEVICE"))
Note: Check this video for more reference about Certificates


Allow for custom handling when the state cookie is missing

**Is your feature request related to a problem? Please describe. **
As described in #298 we have found scenarios where the state cookie might be missing. Adding #300 would also introduce scenarios where state might be available but have expired.
We should handle that scenario in a way that it has some default implementation but is customizable for those who want/needs that.
What area is it related to
BankID
Describe the solution you'd like
Check in BankIdController > Login if the state cookie is still there and is valid. If not, let the injected handler handle it.
if (IsInvalidStateCookie())
{
  if(unprotectedLoginOptions == null) {
    await _bankIdInvalidStateHandler.HandleAsync(context);
  } else {
   await _bankIdInvalidStateHandler.HandleAsync(context, unprotectedLoginOptions.CancelReturnUrl);
  }

    return;
}

A default implementation could use the cancelUrl, like this:
public BankIdCancelUrlInvalidStateHandler : IBankIdInvalidStateHandler {
  public TaskHandleAsync(HttpContext context) {
    context.Response.Redirect("/");
  }

  public TaskHandleAsync(HttpContext context, string cancelReturnUrl) {
    context.Response.Redirect(cancelReturnUrl);
  }
}

Would love to see this. :+1:


chore(docs): Add readthedocs documentation scaffold

This PR adds a basic scaffold for readthedocs.  It gives us a framework in which we can start adding docs.  The config is set to build the this branch (add-rtd), so we need to switch that before merging.
Todo

[ ] Switch config to build master branch
[ ] Revise index.rst and setup basic TOC

Readthedocs can support both reStructuredText and Markdown.  So, if we would rather write in markdown, we can.  Markdown files can't use the extra features of rst, tho, like TOC generation, includes, references, etc.
For example, we could use rst for the main index doc, which will have a TOC.  Then, we could use markdown for other files.
Or, we can just go all-in on rst--might be easier to pick one format and just go with it.

Or, we can just go all-in on rst--might be easier to pick one format and just go with it.

I think just pick one rst and go with it if thats a possibility especially if it has more features, we just probably need a rst cheatsheet when writing docs/et al


Bump Kotlin version

Kotlin 1.0.4 was released a few days ago.
This PR updates the Kotlin version correspondingly.

Duplicate #74


Update 4_06_1 branch to master and add .opam file

This PR adds an .opam file and merges 4_06_1 with master. Unfortunately ppx_optcomp doesn't support OCaml 4.02.3 so this branch still isn't compatible with that version of OCaml.

@wokalski thanks for setting it up.
What's the command to build this with esy/opam?

@cristianoc for opam, I think the standard workflow should work. I haven't used opam in some time now but you should be able to pin and build it like any other package. I tested it together with another project where i used it (I used esy's link feature for that.)

I found a way to build it with esy. It requires adding esy.json with:
{
  "source": "./genType.opam",
  "override": {}
}

Unfortunately, when I build it with esy build it won't build (even though it is built correctly when used as a linked dependency)
info esy build 0.5.6 (using esy.json)
    ocamldep src/.GenType.eobjs/string_map.pp.ml.d (exit 2)
(cd _build/default && /Users/wczekalski/.esy/3______________________________________________________________/i/ocaml-4.7.1004-e5a2754f/bin/ocamldep.opt -modules -impl src/string_map.pp.ml) > _build/default/src/.GenType.eobjs/string_map.pp.ml.d
File "ext/map.cppo.ml", line 7, characters 2-4:
Error: Syntax error
error: command failed: 'dune' 'build' '-p' 'genType' '-j' '4' (exited with 1)
esy-build-package: exiting with errors above...
error: build failed with exit code: 1

esy: exiting due to errors above

@wokalski I get the same error.
Though dune build works fine.
Also, easy takes 1 minute "pause" each time I try to build again, not sure why.
Perhaps it's using a different ocaml version than 4_06_1, not sure.

Looks like it's using 4.7.
(cd _build/default && /Users/cristianoc/.esy/3______________________________________________________________/i/ocaml-4.7.1004-e5a2754f/bin/ocamlc.opt -w -30-40 -g -bin-annot -I src/.GenType.eobjs/byte -I /Users/cristianoc/.esy/3______________________________________________________________/i/ocaml-4.7.1004-e5a2754f/lib/ocaml/compiler-libs -no-alias-deps -o src/.GenType.eobjs/byte/TranslateTypeExprFromTypes.cmo -c -impl src/TranslateTypeExprFromTypes.re.ml)

@cristianoc it seems like an esy bug. I'll report it. In any case, this PR is to the 4_06_1 branch. Can we merge it?
When it comes to proper upstreaming, I am considering writing a simple PPX which will allow conditional compilation just based on the OCaml version. In the current state of affairs, there's nothing that lets us efficiently compile conditionally. ppx_optcomp is 4.04.1+ (because of ppxlib). An alternative would be using .ml files and cppo for everything.

Aha I did not notice it was a PR to the branch.
I would not worry about conditional compilation. There's a difference also in the example, and generated, files. So it would end up being more complexity than it's worth.
I think we'll just switch to support 4.06 at some point, and stop new development on 4.02.3

Great, I'm fine with this 👍. I'll add a comment in the 4.06.1 issue about the branch so that people can use it.


Enable and Force HTTPS on the leafletjs.com website

HTTPS has been released for Github Pages custom domains:
https://blog.github.com/2018-05-01-github-pages-custom-domains-https/
Geolocation examples are not currently working because the site is using http.
Edit 2018-05-03: added explanation for linked blog post.

Yeah. We should do this eventually, but it's not trivial because GitHub doesn't support HTTPS on its custom-domain websites. We'd need to set up some kind of proxy, e.g. with CloudFlare.

Hi @mourner,
Actually if I understand correctly the blog post linked by @theel0ja, GitHub just added support of HTTPS for custom domains, hence why @theel0ja opened this Feature Request?

Damn, apparently I had a case of selective blindness. Sorry! Yes, we should definitely do this now. @JasonSanford the blog says the DNS config should be updated with new IPs, can you take a quick look?

@ghybs Yes, I sent the feature request because Github added HTTPS support for custom domains.

I just updated the A records to point to
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>

The previous TTL was 30 minutes so hopefully soon we'll be hitting the new IP addresses, then we can ✅ it hopefully.

@JasonSanford thanks a lot Jason!
Hmm, I'm seeing leafletjs.com resolving to the new IP addresses, but the browsers still say "Not secure". Here are details from Safari for example:

No idea how to resolve this — I'm really bad at security stuff. Any clues?

Let me try something now...

Mixed content should be fixed.

@theel0ja great! It started working for me for https://leafletjs.com, but for some reason https://www.leafletjs.com times out. Is this an issue on the GitHub side or with our configuration?
Let's fix the mixed content thing — want to do a PR?

@mourner www.leafletjs.com is directed to a Namecheap IP address.
https://iplookup.theel0ja.info/?host=www.leafletjs.com

@JasonSanford does the above mean we should add some records on the DNS config side for www specifically?

That sounds right. Let me look in the DNS config.

The DNS settings had the www subdomain redirecting to http, not https. I've changed this and it should be propagating now.

@JasonSanford hmm, will a redirect work with https? It seems that it wouldn't. Could we just add records for www to point to GitHub IPs just like for the core domain?

https://github.com/Leaflet/Leaflet/pull/6150

Merged the mixed content fixes. https://www.leafletjs.com is still not working — let's wait for a fix to that, and then I'll be able to turn on "enforce https" in the settings.

I mess around with DNS about once a year and always struggle. What I've got below is apparently not working. Anyone have ideas?

Here's the rest of the records for reference.

@JasonSanford found this GitHub help page which suggests pointing CNAME to the GitHub domain, e.g. leaflet.github.io — not sure if it's applicable in our case but maybe worth a try.

We're now enforcing https, and also www.leafletjs.com redirects to https://leafletjs.com, which is good enough to close this issue. The only remaining thing is that https://www.leafletjs.com doesn't work, but let's leave it at that and maybe find a solution later.

Hmm suddenly https://www.leafletjs.com started to work — no idea what changed (maybe some cache updated) but great :) @JasonSanford no further action needed then.


Wofklow Pattern

Limit trigger on release pipeline to pushes for tags with specific pattern.

Pull Request Test Coverage Report for Build 326348308

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 76.709%

Totals

Change from base Build 326341093:
0.0%

Covered Lines:
1212

Relevant Lines:
1580

💛  - Coveralls


Signer/Aggregator nodes sign only when Cardano node is synchronized

Issue
The signer and aggregator nodes should not start signing and gathering stake distribution until the underlying Cardano node is (almost) fully synced

Since we have implemented the era mechanism, the behavior of the aggregator node at startup has changed and partially mitigated this issue: the era reader reads an utxo on the chain which is not available, which makes the node panic very early.


fix: Splash screen shown once from Welcome Fragment

Fixes #993
Changes:
Used Navigation Controller along with passing a value to prevent restarting activity.
Screenshots for the change:

@liveHarshit @nikit19 Your review.

@liveHarshit Create an issue to create type safe intents using navigation architecture components

PR should close now, no explicit intents are there.


WIP downgrade eslint-plugin-promise to 5.2.0

to fix peerDependencies issue with npm >= 8.5.5

Messages

:book:
:sparkles: Thanks for your contribution to Shields, @chris48s!

  Generated by :no_entry_sign: dangerJS against 736e2c2eac2abe443c4de70de9cf1e46eb569324


[dep-bot] Updated github.com/goccy/go-json to v0.10.2

Updated github.com/goccy/go-json to v0.10.2

This PR has been merged into update-dependencies. Thanks for your contribution!

This PR has been merged into update-dependencies. Thanks for your contribution!

This PR has been merged into update-dependencies. Thanks for your contribution!


Non default user model and Active Storage has_one_attached

Having a basic Rails 5.2 app with only clearance, Active Storage, and a Member model. The Member model is set as user_model in an initializer.
Clearance.configure do |config|
  config.mailer_sender =<EMAIL_ADDRESS>  config.rotate_csrf_on_sign_in = true
  config.user_model = Member
end

and Member model:
class Member < ApplicationRecord
  include Clearance::User
  has_one_attached :avatar
end

Then the rails app isn't able to start the server due to a circular dependency. All is fine when the user model is the default or when has_one_attached isn't in the model. How to fix this?

What is the full error you get?

 $ rails s
=> Booting Puma
=> Rails 5.2.0 application starting in development
=> Run `rails server -h` for more startup options
Exiting
Traceback (most recent call last):
	104: from bin/rails:3:in `<main>'
	103: from bin/rails:3:in `load'
	102: from /Users/lethan/Development/Homepages/clearance_test/bin/spring:15:in `<top (required)>'
	101: from /Users/lethan/Development/Homepages/clearance_test/bin/spring:15:in `require'
	100: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/lib/spring/binstub.rb:31:in `<top (required)>'
	 99: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/lib/spring/binstub.rb:31:in `load'
	 98: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/bin/spring:49:in `<top (required)>'
	 97: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/lib/spring/client.rb:30:in `run'
	 96: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/lib/spring/client/command.rb:7:in `call'
	 95: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/lib/spring/client/rails.rb:28:in `call'
	 94: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/spring-2.0.2/lib/spring/client/rails.rb:28:in `load'
	 93: from /Users/lethan/Development/Homepages/clearance_test/bin/rails:9:in `<top (required)>'
	 92: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:283:in `require'
	 91: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:249:in `load_dependency'
	 90: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:283:in `block in require'
	 89: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:29:in `require'
	 88: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:20:in `require_with_bootsnap_lfi'
	 87: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:65:in `register'
	 86: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `block in require_with_bootsnap_lfi'
	 85: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require'
	 84: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands.rb:18:in `<main>'
	 83: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/command.rb:46:in `invoke'
	 82: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/command/base.rb:65:in `perform'
	 81: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
	 80: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
	 79: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
	 78: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands/server/server_command.rb:142:in `perform'
	 77: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands/server/server_command.rb:142:in `tap'
	 76: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands/server/server_command.rb:147:in `block in perform'
	 75: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands/server/server_command.rb:51:in `start'
	 74: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands/server/server_command.rb:89:in `log_to_stdout'
	 73: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/server.rb:354:in `wrapped_app'
	 72: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/commands/server/server_command.rb:27:in `app'
	 71: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/server.rb:219:in `app'
	 70: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/server.rb:319:in `build_app_and_options_from_config'
	 69: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/builder.rb:40:in `parse_file'
	 68: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/builder.rb:49:in `new_from_string'
	 67: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/builder.rb:49:in `eval'
	 66: from config.ru:in `<main>'
	 65: from config.ru:in `new'
	 64: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/builder.rb:55:in `initialize'
	 63: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/rack-2.0.5/lib/rack/builder.rb:55:in `instance_eval'
	 62: from config.ru:3:in `block in <main>'
	 61: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:44:in `require_relative'
	 60: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:283:in `require'
	 59: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:249:in `load_dependency'
	 58: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:283:in `block in require'
	 57: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:29:in `require'
	 56: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:20:in `require_with_bootsnap_lfi'
	 55: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:65:in `register'
	 54: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `block in require_with_bootsnap_lfi'
	 53: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require'
	 52: from /Users/lethan/Development/Homepages/clearance_test/config/environment.rb:5:in `<main>'
	 51: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/application.rb:361:in `initialize!'
	 50: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/initializable.rb:60:in `run_initializers'
	 49: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:205:in `tsort_each'
	 48: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:226:in `tsort_each'
	 47: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:347:in `each_strongly_connected_component'
	 46: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:347:in `call'
	 45: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:347:in `each'
	 44: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:349:in `block in each_strongly_connected_component'
	 43: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:415:in `each_strongly_connected_component_from'
	 42: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:415:in `call'
	 41: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/initializable.rb:50:in `tsort_each_child'
	 40: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/initializable.rb:50:in `each'
	 39: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
	 38: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:431:in `each_strongly_connected_component_from'
	 37: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
	 36: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
	 35: from /Users/lethan/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/tsort.rb:228:in `block in tsort_each'
	 34: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/initializable.rb:61:in `block in run_initializers'
	 33: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/initializable.rb:32:in `run'
	 32: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/initializable.rb:32:in `instance_exec'
	 31: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/engine.rb:613:in `block in <class:Engine>'
	 30: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/engine.rb:613:in `each'
	 29: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/engine.rb:614:in `block (2 levels) in <class:Engine>'
	 28: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/engine.rb:656:in `load_config_initializer'
	 27: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/notifications.rb:170:in `instrument'
	 26: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/railties-5.2.0/lib/rails/engine.rb:657:in `block in load_config_initializer'
	 25: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:277:in `load'
	 24: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:249:in `load_dependency'
	 23: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:277:in `block in load'
	 22: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:50:in `load'
	 21: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:50:in `load'
	 20: from /Users/lethan/Development/Homepages/clearance_test/config/initializers/clearance.rb:1:in `<main>'
	 19: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/clearance-1.16.1/lib/clearance/configuration.rb:202:in `configure'
	 18: from /Users/lethan/Development/Homepages/clearance_test/config/initializers/clearance.rb:4:in `block in <main>'
	 17: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:193:in `const_missing'
	 16: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/active_support.rb:43:in `load_missing_constant'
	 15: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:502:in `load_missing_constant'
	 14: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:348:in `require_or_load'
	 13: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:37:in `load_interlock'
	 12: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies/interlock.rb:13:in `loading'
	 11: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/concurrency/share_lock.rb:151:in `exclusive'
	 10: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies/interlock.rb:14:in `block in loading'
	  9: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:37:in `block in load_interlock'
	  8: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:365:in `block in require_or_load'
	  7: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:467:in `load_file'
	  6: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:653:in `new_constants_in'
	  5: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/activesupport-5.2.0/lib/active_support/dependencies.rb:468:in `block in load_file'
	  4: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:100:in `load'
	  3: from /Users/lethan/.rvm/gems/ruby-2.5.1/gems/bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:100:in `load'
	  2: from /Users/lethan/Development/Homepages/clearance_test/app/models/member.rb:1:in `<main>'
	  1: from /Users/lethan/Development/Homepages/clearance_test/app/models/member.rb:3:in `<class:Member>'
/Users/lethan/.rvm/gems/ruby-2.5.1/gems/activerecord-5.2.0/lib/active_record/dynamic_matchers.rb:22:in `method_missing': undefined method `has_one_attached' for Member (call 'Member.connection' to establish a connection):Class (NoMethodError)

It was a bug in Active Storage, that now has been fixed.

@lethan I'm getting this error with Rails <IP_ADDRESS>.  Please could you link to the ActiveStorage commit that fixes this?


Receiving messages only after a very long time

Hello, my problem has been that when I establish a connection to my websocket server and begin sending messages, the messages can all be sent, but at some point I stop receiving the messages I am supposed to receive, or rather more accurately, the messages only arrive after a very long time.
I believe that it has something to do with my application code, as if something is blocking the receiving queue. I have tested this with other clients that were able to communicate with the server and they all seem to do just fine.
Nonetheless, other websockets in my application function fine whilst this one isn't able to receive any messages:
    async def _connect(self):
        backoff = ExponentialBackoff(2)
        while not (self.ws and self.ws.open):
            try:
                self.ws = await websockets.connect(self.uri, extra_headers=self.headers)
                task = asyncio.create_task(self.listen())
                # asyncio.create_task(hello())
                self.keep_alive = KeepAlive(self, 3)
                self.keep_alive.start()
                # await task
            except OSError:
                delay = backoff.delay()
                logger.error(f"Connection refused, trying again in {delay:.2f}s")
                await asyncio.sleep(delay)

    async def listen(self):
        try:
            while True:
                msg = await self.ws.recv()
                logger.debug(f"Received websocket message from `{self.name}`: {msg}")
                await self.on_message(json.loads(msg))
        except websockets.ConnectionClosed:
            pass  # ping() handles this for us, no need to hear it twice..

    async def send(self, msg):
        if not self.ws or not self.ws.open:
            self.available = False
            raise NodeException("Websocket is not ready, cannot send message")
        logger.debug(f"Sending websocket message: {msg}")
        await self.ws.send(json.dumps(msg))

What am I doing wrong here?

I'm not seeing anything obvious here. Enabling debug logs might help: https://websockets.readthedocs.io/en/stable/cheatsheet.html#debugging
Running your own keepalive task doesn't seem necessary. Instead you should adjust the parameters of websockets' built-in keepalive if they don't do what you want.
Based on the symptoms you're describing, you might be saturating the connection from the client to the server. In that case websockets will apply backpressure and stop processing input until the output gets better. Specifically, when it receives a ping, it will wait until it can send the corresponding pong, and it can't as long as the send buffers are full. Most libraries don't handle backpressure correctly, which could explain why websocket stands out of the pack.
This is fairly uncommon, though: it requires sending a lot of data faster than the server can process it. In fact, it's somewhat tricky to trigger on purpose. Usually things break down before backpressure comes into play.

Is there a way to monitor how much my websockets are sending on a lower level, or even verify that it is due to backpressure? I've tried to use the debug settings, but it doesn't seem manageable to look at a million messages being sent all at once.

If you're sending a million messages at once, perhaps you're facing the issue I suggested.
websockets doesn't do anything in this regards. Everything is handled by asyncio and then the operating system (and then the NIC, if you go all the way down...)
As far as I know asyncio doesn't provide a way to inspect the send buffer, but you should see something in the logs if you saturate it:
https://github.com/python/cpython/blob/033d10bd21d962a59c6c4fc503092046baa451a1/Lib/asyncio/streams.py#L158
You can try experimenting with https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.set_write_buffer_limits. If you set it higher than the total amount of data your application may send "at once", is it better? e.g. if you're sending one million messages of 32 bytes each, add 8 bytes for the WebSocket header, and set the buffer to 40 megabytes.
(I don't know how well Python will handle a 40MB bytes object. This may grind the interpreter to a halt.)
Keep in mind that the OS may buffer several megabytes as well. Controlling the asyncio write buffer doesn't give you full control. However, it can give you a general idea  of what's happening.

Hrm, wait, scratch that — websockets no longer uses asyncio's StreamWriter. It writes straight to the transport. So all you have is the OS buffer.
In general, this should be more efficient, but it may degrade less gracefully under excessive write load (our theory here).
Furthermore, the OS buffers usually much larger than asyncio.StreamWriter's buffer: a few MB vs 64kB if memory serves, so I don't think missing the asyncio buffer makes a big difference in your situation.

You can try commenting out this block of code:
https://github.com/aaugustin/websockets/blob/69c94af5c0ad19402e0bedcc6b61a23fa070c946/src/websockets/protocol.py#L1069-L1081
If it helps, then it means you're indeed saturating the write buffers.

If you need to write a lot of data much faster than you can send it to the network, you can try increasing the buffer size with setsockopt on SEND_BUF_SIZE.

I cannot remember what I wanted to document so I am going to close this issue.


LightStreamer disconnects after about 2 hours

This was already raised years ago in #84 and a possible fix was made in #113 but the issue still occurs. I am creating a new ticket so it can be investigated again, perhaps a different approach is needed to finally fix this issue. Pinging @sacharbit who provided the initial fix.
This is the error I am seeing:
ERROR:trading_ig.lightstreamer:Communication error
Will re-run with logging in DEBUG mode, wait... 2 hours and see if any more info is reported.

I made some progress on my implementation of a heartbeat monitor in stream.py (IGStreamService). What it does is if the heartbeat stops pinging for more than 10 seconds, it will assume connection dropped, and retry to connect. To do so it will need to re-subscribe to all subscriptions once connection is re-established. Issue with this I think is the stream update thread is re-spawned and therefore so are listener methods. I can see two ways of implementing this in a non-disruptive way:

Have a method in IGStreamService e.g. connection_alive() that returns False in case of a drop in connection. This requires the main thread to have a while loop i.e. while stream.connection_alive(). It's good in that it enables the user to choose how to handle a disconnect and re-subscribe if they so wish. I added a resubscribe_all() method to stream.py to facilitate this
Automatically reconnect, but doing so from stream.py won't work for the reason mentioned above. I think we'd need to intercept a disconnection in lightsreamer.py's _receive() but I'm not too keen to touch that since it's obscure at best and the chances of unexpected behaviour are high

Any thoughts @oliverpolden @bug-or-feature @femtotrader ? Good thing is, since the (old) code samples encouraged to reference lighstreamer.py's subscribe() directly, adding the above will not break existing code. However going forward it would require to use stream.py's subscribe method instead since it keeps track of active subscriptions, enabling it to resubscribe_all if required.

Also pinging @sacharbit who PRd the original fix

It's been 2 or 3 years since I used that library so I don't really remember but I know I've ran into the same issue during the weekend since there was no data for a long time. But it didn't bother me because in the weekend there's no data anyway and I was using it for very liquid products so there's always new data coming in less than 10 secs. I guess for more exotic ones, you'll need to have a workaround. Can't really help you with that, it took me waaaaay too long to figure out that fix that i've PRed and couldn't find anything else!

Sorry - I don't use the lightstreamer code

@RRMXkun - Did you ever come up with a working solution to this issue?

@RRMXkun - Did you ever come up with a working solution to this issue?

yes, but it's not perfect in that it requires the main thread to loop. I basically have a connection_alive() method in IGStreamService which switches to False and throws an exception if the Lightstreamer Heartbeat stops responding for 5 seconds. I didn't want to push the code because I haven't been able to test it for long enough.

@RRMXkun - Did you ever come up with a working solution to this issue?

yes, but it's not perfect in that it requires the main thread to loop. I basically have a connection_alive() method in IGStreamService which switches to False and throws an exception if the Lightstreamer Heartbeat stops responding for 5 seconds. That means the main thread needs to be running a while loop to check for the value returned by connection_alive(). However the plus side is, the user is able to catch that exception and decide what to do.
I didn't want to push the code because I haven't been able to test it for long enough. I wanted to add a nice auto reconnect option but it's tricky to test.

Thanks @RRMXkun  I've been trying something similar but my 'keep alive' check is in the high level calling code and checks for a flag written to a database (could just be a file) - My check loop replaces the INPUT section of IG Markets Stream API sample from Femto. I have then modifiied Lighstreamer.py to write the flag to the database/file, if there is any type of error. This allows the check to be seperate from the main thread. It's not an ideal solution though and I'm still testing.

@RRMXkun - Did you ever come up with a working solution to this issue?

yes, but it's not perfect in that it requires the main thread to loop. I basically have a connection_alive() method in IGStreamService which switches to False and throws an exception if the Lightstreamer Heartbeat stops responding for 5 seconds. That means the main thread needs to be running a while loop to check for the value returned by connection_alive(). However the plus side is, the user is able to catch that exception and decide what to do.
I didn't want to push the code because I haven't been able to test it for long enough. I wanted to add a nice auto reconnect option but it's tricky to test.

Thanks @RRMXkun I've been trying something similar but my 'keep alive' check is in the high level calling code and checks for a flag written to a database (could just be a file) - My check loop replaces the INPUT section of IG Markets Stream API sample from Femto. I have then modifiied Lighstreamer.py to write the flag to the database/file, if there is any type of error. This allows the check to be seperate from the main thread. It's not an ideal solution though and I'm still testing.

I see, has your additional logging provided any insights as to what might be happening? From my end I've got things working relatively, I think having an exception thrown on the main thread is actually the right thing to do since it enables handling as per the user's wish. Of course being able to silently reconnect would be better, but the only way I managed to do that so far is by re-subscribing everything. Problem is this spawns a new set of threads, so any reference made to the old, now-dead ones result in a Nonetype error. Once I manage to get that bit to work I will PR.
If you've made some progress your end please share, you may have come up with a better solution

I'm closing this, it's old. The latest release uses the official Lightstreamer Python client SDK, which does auto-reconnections, auto-resubscriptions. See #302


Contribution guide

Markdown file outlining how to contribute to the project

@Atraya21 What do you have in mind?


about the integration of 'Remote LMDB' PR

Is this PR  just merged and wait to be applied globally yet?

the source PR goes as #21


Fixed matplotlib HeatMap extents

HeatMap extents weren't being set correctly when the values were numeric.

[x] Adds unit test

So I think matplotlib HeatMaps have been broken for quite a while, this needs a test data update.


Missing production files version 2.4.35

We get a new version 2.4.35 but oh surprise! cannot find linq.min.js and linq.js from dist folder
We use this package for backend and frontend but
for web -> https://unpkg.com/linq-es2015/dist/linq.min.js
We get a Cannot find "/dist/linq.min.js" in<EMAIL_ADDRESS>really is last version
but<EMAIL_ADDRESS>works fine, files exist :)
dist folder

Production files

In new version those files doesn't exist

we go back to version 2.4.34 meanwhile
can we wait for some solution? or you recommend migrate our code importing from source and no-minified version from folder /lib?

Fixed in 2.4.36


[OTEL-2101] Start ddflare extension even if there is no DD exporter config

What does this PR do?
Start ddflare extension even if there is no DD exporter config
Motivation
Describe how to test/QA your changes
Will be covered in the E2E test
Possible Drawbacks / Trade-offs
Additional Notes

/merge


feat(net): Bandwidth monitoring

Addresses #570, taking heavy inspiration from https://github.com/libp2p/rust-libp2p/blob/master/src/bandwidth.rs
Adds support for monitoring bandwidth in terms of the number of inbound & outbound bytes on streams established w/ the network

Ref https://github.com/paradigmxyz/reth/issues/140 as well

@mattsse should I apply the usage of the MeteredStream throughout the codebase in this PR? If so, would love some pointers on where! (It's also kinda why the tests are failing atm, because the MeteredStream::new isn't used anywhere 😅 )

Codecov Report

Merging #707 (33ceb34) into main (ade1ea4) will increase coverage by 0.12%.
The diff coverage is 93.60%.

@@            Coverage Diff             @@
##             main     #707      +/-   ##
==========================================
+ Coverage   72.68%   72.80%   +0.12%
==========================================
  Files         268      269       +1
  Lines       27604    27771     +167
==========================================
+ Hits        20063    20219     +156
- Misses       7541     7552      +11

Flag
Coverage Δ

unit-tests
72.80% <93.60%> (+0.12%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

crates/net/common/src/lib.rs
100.00% <ø> (ø)

crates/net/network/src/session/handle.rs
50.00% <ø> (ø)

crates/net/common/src/bandwidth_monitor.rs
92.85% <92.85%> (ø)

crates/net/network/src/manager.rs
49.78% <100.00%> (+0.43%)
:arrow_up:

crates/net/network/src/network.rs
63.86% <100.00%> (+0.61%)
:arrow_up:

crates/net/network/src/session/active.rs
83.50% <100.00%> (+0.10%)
:arrow_up:

crates/net/network/src/session/mod.rs
69.41% <100.00%> (+0.26%)
:arrow_up:

crates/net/eth-wire/src/types/version.rs
77.77% <0.00%> (-2.23%)
:arrow_down:

crates/net/eth-wire/src/capability.rs
61.79% <0.00%> (-1.13%)
:arrow_down:

crates/stages/src/stages/bodies.rs
93.55% <0.00%> (+0.39%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@mattsse I think it's ready! Let me know what you think 🙏

@mattsse thank you for the feedback, as always! good to merge now?


Fix mkdocs autodoc link to log_and_django_terminal in contributing.md

A links within a contributing.md note of example tests of log_and_django_terminal broke.

Fixed by 61716df9814afff4f4d811d7dfec8a7053a99846 (possibly before).


Filter suppressClassNameWarning to not to pass down to wrapped components

fixes #2346
Once I tried the solution that @leantide mentioned in the related issue, but it didn't work (I think it is because React likely freezes props).
What I did Instead is to filter suppressClassNameWarning when passing props to child component.

Updated! Thanks @mxstbr for taking a look!

Is there a next step on this?

I'm not sure.
@mxstbr Is there anything left to do?


Update Wikidata support and mapping

[x] https://github.com/citation-js/citation-js/issues/152 (see also #160)
[ ] https://github.com/citation-js/citation-js/issues/135
[ ] https://github.com/citation-js/citation-js/issues/106
[x] https://github.com/citation-js/citation-js/issues/46
[ ] Evaluate support for software, datasets, and material samples
[ ] https://github.com/larsgw/citation.js/issues/67

Example: https://www.wikidata.org/wiki/Q102040491 (https://query.wikidata.org/#SELECT %3Fitem %3FitemLabel %3Fedition %3FeditionLabel WHERE {
  %3Fitem wdt%3AP31%2Fwdt%3AP279* wd%3AQ7397 %3B wdt%3AP747 %3Fedition .
  SERVICE wikibase%3Alabel { bd%3AserviceParam wikibase%3Alanguage "[AUTO_LANGUAGE]%2Cen". }
})

Note that while  has edition or translation (P747)  is used for things like books or ontologies, software versioning is expressed via the dedicated property  software version identifier (P348) .


Unhandled lua error when no available plugins in configuration

I was running Kong (from the mashape/kong:latest Docker container) on Mesos with a malformed config, specifically, one that had no plugins_available field. I got this error:
/usr/local/bin/luajit: /usr/local/share/lua/5.1/kong/tools/migrations.lua:55: bad argument #1 to 'ipairs' (table expected, got nil)
stack traceback:
	[C]: in function 'ipairs'
	/usr/local/share/lua/5.1/kong/tools/migrations.lua:55: in function 'migrate_all'
	/usr/local/share/lua/5.1/kong/cli/utils/signal.lua:174: in function 'prepare_database'
	/usr/local/share/lua/5.1/kong/cli/utils/signal.lua:225: in function 'prepare_kong'
	/usr/local/share/lua/5.1/kong/cli/start.lua:21: in main chunk
	[C]: in function 'require'
	/usr/local/lib/luarocks/rocks/kong/0.5.3-1/bin/kong:55: in main chunk
	[C]: at 0x00404ac0

Looks like the error happens here. After putting this field in my config, I no longer saw this error.
While the Kong docs say that every field is required, this is not a particularly helpful error message!

Yes, that is taken care of by #596 already, and in next, which will be out with the next feature-release (0.6). Thanks


Organize Fate Operations

The RepOs of every fate operation are ALL in the org.apache.accumulo.master.tableOps package.  It is annoying to track down which RepO goes with which operation.  Creating a package in o.a.a.master.tableOps for each operation would organize them nicely.  This would have to be done in 2.0 since it will break serialization.

@milleruntime Am I right by saying you want these sorted by the operations like
CREATE, DELETE, RENAME, SET_PROPERTY, REMOVE_PROPERTY, ETC.
Or do you want them grouped like CloneInfo, CloneMetadata, ClonePermissions, CloneTable, CloneZookeeper, FinishCloneTable all put into a subpackage say clone and so on for the rest of them for their respective operation.

Check out bulkver1 and bulkver2
It would be like those packages but one for each operation in FateOperation

So TABLE_BULK_IMPORT and TABLE_BULK_IMPORT2 are already done, that is what bulkver1 and bulkver2 cover.

It would probably be easiest to start with one FateOperation and open a pull request first to make sure we are on the same page.  Then you could do a second PR for the rest.


Bring back the "bug" of rounded corners not restoring after unsnapping.

Description of the new feature / enhancement
In 0.57.2, one of the changes was "#17406: FancyZones not restoring rounded corners after unsnapping bug is fixed." I really liked that window corners stayed sharp. I thought this was a feature not a bug and I am disappointed that it was fixed. Please bring it back. :)
Scenario when this would be used?
For windows 11 users that don't like the rounded corners because it leaves unnecessary gaps between windows and it looks ugly.
Supporting information
No response

This should be a setting now. Can you double check? I don't have Win.11 myself.

The setting is for removing rounded corners when snapped to a zone, this works perfect. Before the newest update, windows would stay with sharp corners after unsnapping. Now, they go back to rounded. I can't find any setting that makes the sharp corners stay after unsnapping.

If you want sharp corners outside of FancyZones, I think the Windows Feedback Hub is the place to be.


Update http links

@NextTurn : Thanks for your contribution! The author, @alkohli, has been notified to review your proposed change.

Verified link fixes.


Market Research

Additional market research is required to ensure we are building a product that the consumer (or business) needs/wants.
Starting with designers, or makers.
Also what spectrum of materials they use and where they obtain.

Scope of project has changed as of 10/2014.  Topic CLOSED


Fix theme logic and rebuild prod assets

The JS changes were already reviewed, just not "built". :+1:

Sorry, my bad. I assumed building the JS was only needed when the theme is released, not when filing or merging the PR.


Issue when upgrading to ASP.Net Core 2.0.2

Hi All,
I've recently upgraded to ASP.NET Core 2.0.0 and now I'm getting the following error:
System.MissingMethodException: 'Method not found: 'Void Microsoft.Net.Http.Headers.EntityTagHeaderValue..ctor(System.String)'.'
Decompiling the method, it looks like its constructor's signature changed and is now:
public EntityTagHeaderValue(StringSegment tag)

Dup of #408


Update btstack v1.6.1

fyi @mringwal This seems to work for develop as well

people should not downgrade BTstack within Pico SDK

I agree. The issue here is we allow clients to use their own btstack folder in their project. So it's not that they are downgrading, just that they haven't (yet) upgraded.


Seeing HTTP 401 on an XHR request while not using Portal (the portal JavaScript is still being included)

Issue Summary
Since upgrading to Ghost v4 on a self-hosted instance, a JavaScript related to Portal is being loaded (included in {{ghost_head}} by Ghost itself), while Portal is not used and disabled (ie. not configured).
This JavaScript is loaded from https://unpkg.com/<EMAIL_ADDRESS>which is not needed in this case, and attempts to perform an XHR to /blog/members/api/member/ that returns HTTP 401 (Cookie ghost-members-ssr not found).
I've investigated, and found out that the Portal JS is unconditionally included in ghost_head. It should only be included if Portal is in use (and as well, allow to use a self-hosted JavaScript if possible, for OSS installs).
Note that Google generates SEO warnings due to the XHR request reporting as 401, which is how I found out about this newly included script.
Code references:

JS code generator: https://github.com/TryGhost/Ghost/blob/main/core/frontend/helpers/ghost_head.js#L39
JS code include: https://github.com/TryGhost/Ghost/blob/main/core/frontend/helpers/ghost_head.js#L181
JS resource URL: https://github.com/TryGhost/Ghost/blob/main/core/shared/config/defaults.json#L123

Indirectly related: #12778
To Reproduce

Disable Portal on a theme with {{ghost_head}} support
Inspect the blog pages, which will contain <script defer src="https://unpkg.com/<EMAIL_ADDRESS>data-ghost="https://crisp.chat/blog/"></script> (which we do not want here)

I would expect the Portal script not being included in the case Portal is not configured.
Technical details:

Ghost Version: v4.1.0
Node Version: v10.21.0
Browser/OS: (does not apply)
Database: (does not apply)

meanwhile you can patch the mentioned default.json setting the url key in line 124 to an empty string (or an empty script).
This downgrades the error to a warning.

The portal script is required, whether or not you are using the portal UI as it also handles other JavaScript hooks in the theme.  Ghost doesn't know if your theme is or isn't using those hooks so the script is always required. Therefore this is expected behaviour, not a bug.

I see, thanks for the reply. Then I guess it'd be good that the /blog/members/api/member/ HTTP route called return a 2xx status code even if not configured, as it's seemingly ringing alerts on Google's SEO tools, and other tools.
Which I think is a non-issue report you'll receive on a recurring basis from Ghost or Ghost Pro users.

We are looking at ways to mitigate the 401, but it's a slightly different issue :)

@ErisDS I feel the need to get out some frustrations here that I hope in return will help you when developing new features going forward. Introducing Membership is a great addition to the product but not something that we as a company are planning to use. My guess is that most companies won't since they earn their revenue elsewhere and the blog is there to support their business only and not be responsible for revenue. That's why I believe that from one major to another introducing a new large (almost 500 kb!) script is a bad practice. As described here the script is used for other features than just Membership which requires a lot of testing on our part to figure out if it can be removed or not (or replaced with an empty script to avoid Ghost generating invalid HTML). Also, the script is injected directly into the head section of the markup. We have built a custom theme and reference all JavaScripts just before the end body tag. As already suggested by other users, having a script this large and then in the head, completely destroys the Lighthouse score.
I hope that you can use the feedback here. Introducing stuff like this will definitely make me think twice about upgrading to a new major next time. If introducing new features like this, at least let us opt-in or out from them by separating core stuff from the feature stuff and only include the feature-specific code if enabled through the settings.


Updated s_changeShadowValue method

Updated s_changeShadowValue method to construct updateShadowRequest using ShadowState instead of using completely JsonObject
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Oh... good catch, I didn’t realize we were doing that. You likely need to run clang-format though for the build to pass.

Oh... good catch, I didn’t realize we were doing that. You likely need to run clang-format though for the build to pass.

@JonathanHenson I do not know how to run clang-fromat for the build to pass, can you help here.

[Container] 2019/06/19 16:35:05 Running command ./format-check.sh
samples/shadow/shadow_sync/main.cpp failed clang-format check.
[Container] 2019/06/19 16:35:06 Command did not exit successfully ./format-check.sh exit status 1
[Container] 2019/06/19 16:35:06 Phase complete: BUILD State: FAILED
Assuming you have clang-format-6.0 or later installed:
clang-format -i samples/shadow/shadow_sync/main.cpp

[Container] 2019/06/19 16:35:05 Running command ./format-check.sh
samples/shadow/shadow_sync/main.cpp failed clang-format check.
[Container] 2019/06/19 16:35:06 Command did not exit successfully ./format-check.sh exit status 1
[Container] 2019/06/19 16:35:06 Phase complete: BUILD State: FAILED
Assuming you have clang-format-6.0 or later installed:
clang-format -i samples/shadow/shadow_sync/main.cpp

Thank you. I installed clang-format-6.0 and run the sample with that.

Hi @JonathanHenson ,
Now it passed all the builds and checks, please review it.
Thank you.