Python3

Add Python3 support via machida3 for the following platforms:

install from source (manual)
using docker
vagrant for (for any target distro that's in ubuntu/xenial+, deban/stretch+)
install via wallaroo-up.sh for any distro that's in ubuntu/xenial+, debian/stretch+

Includes:

All python examples are now tested with python3 in addition to the existing python2 tests
Topology tests for python3
Resilience tests for python3
Autoscale tests for python3
Documentation (git book)
Documentation (examples/python READMEs)

While MacOS is not officially supported, @slfritchie did put in a fix in machida3/Makefile that should make it build successfully for from-source installations.

@JONBRWN Are the node errors related at all to https://travis-ci.org/WallarooLabs/wallaroo/builds/445377174#L960 where a binary package download for npm-sass fails (in both Travis-CI checks)?

@slfritchie yes, from what I gathered that's due to Node 11. There's an open PR to add support: https://github.com/sass/node-sass/pull/2522
If it's not resolved by morning, I'll update things to install an older version of node so we can move forward

whoever merges this: please make sure to NOT squash this any further! Just rebase and merge please!

@caj-larsson it's in master! Thanks again for kicking this off!
We're going to start working on the release testing now, with the public release to follow soon.

it's in master
Fantastic! I have a project I started where I was doing feature engineering for a realtime ML-pipeline in the python3 branch. Will switch and write something when I find the time!

@caj-larsson I'd love to hear more about that realtime ML-pipeline. If you have the time and want to chat, my email is<EMAIL_ADDRESS>


Can we allow users to control final newline and trim whitespace independent of save without formatting

https://github.com/microsoft/vscode/issues/38695 had introduced the option of save without formatting which is very useful in working on projects which don't already have a formatter enabled and keeps diffs small and relevant to the lines being changed.
Problem:
But I had assumed that "files.insertFinalNewline": true, was an independent feature which can be turned on or off. But I realized when I found the above issue that save without formatting was designed to silently disable other flags as well. This leads to a situation where I have to remember to add a newline manually when saving without formatting. Recently I found a workaround in the form of an extension which does exactly what I want, so I have a working solution (at least until that extension breaks for some future version), but I believe giving user control over what gets disabled as part of save without formatting would help, or at least visibility into what it disables (so that there are no surprises, because for sometime I suspected it was because of the vscode-neovim plugin).
Other considerations:

I didn't want to disable formatting altogether for the project because for new files in the same project I would like to save (with formatting).
Or maybe format file can become a separate option that users can explicitly call (similar to trim trailing whitespaces). That will also allow me flexibility to opt-in to formatting (as a special feature) rather than it being done implicitly together with newline and trailing whitespace.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


[Python] ERROR: Failed building wheel for pyarrow on Linux

Describe the bug, including details regarding any error messages, version, and platform.
Pyarrow stopped building in our CI at some point a few hours ago (though was building this morning) with error ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects
Container source reference: https://github.com/ucsb-pstat/pstat-174-container-image
This is based on Jupyter upstream base images for Ubuntu 22.04.
I believe this is a duplicate of #36963 but was asked to file a separate issue since this isn't FreeBSD.
      -- Running cmake for pyarrow
      cmake -DPYTHON_EXECUTABLE=/opt/conda/bin/python3.11 -DPython3_EXECUTABLE=/opt/conda/bin/python3.11 "" -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PARQUET_ENCRYPTION=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-pr092oko/pyarrow_c75dd9cd7746480bb33e6fef1767af9b
      -- The C compiler identification is GNU 11.4.0
      -- The CXX compiler identification is GNU 11.4.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- System processor: x86_64
      -- Performing Test CXX_SUPPORTS_SSE4_2
      -- Performing Test CXX_SUPPORTS_SSE4_2 - Success
      -- Performing Test CXX_SUPPORTS_AVX2
      -- Performing Test CXX_SUPPORTS_AVX2 - Success
      -- Performing Test CXX_SUPPORTS_AVX512
      -- Performing Test CXX_SUPPORTS_AVX512 - Success
      -- Arrow build warning level: PRODUCTION
      Using ld linker
      Configured for RELEASE build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
      -- Build Type: RELEASE
      -- Generator: Unix Makefiles
      -- Build output directory: /tmp/pip-install-pr092oko/pyarrow_c75dd9cd7746480bb33e6fef1767af9b/build/temp.linux-x86_64-cpython-311/release
      -- Found Python3: /opt/conda/bin/python3.11 (found version "3.11.4") found components: Interpreter Development.Module NumPy
      -- Found Python3Alt: /opt/conda/bin/python3.11
      -- Found PkgConfig: /opt/conda/bin/pkg-config (found version "0.29.2")
      -- Could NOT find Arrow (missing: Arrow_DIR)
      -- Checking for module 'arrow'
      --   No package 'arrow' found
      CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Arrow (missing: ARROW_INCLUDE_DIR ARROW_LIB_DIR
        ARROW_FULL_SO_VERSION ARROW_SO_VERSION)
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
        cmake_modules/FindArrow.cmake:450 (find_package_handle_standard_args)
        cmake_modules/FindArrowPython.cmake:46 (find_package)
        CMakeLists.txt:231 (find_package)

      -- Configuring incomplete, errors occurred!
      See also "/tmp/pip-install-pr092oko/pyarrow_c75dd9cd7746480bb33e6fef1767af9b/build/temp.linux-x86_64-cpython-311/CMakeFiles/CMakeOutput.log".
      error: command '/usr/bin/cmake' failed with exit code 1
      [end of output]

Component(s)
Python

If you want to use conda, you should use https://anaconda.org/conda-forge/pyarrow .
If you don't want to use https://anaconda.org/conda-forge/pyarrow , you can use https://anaconda.org/conda-forge/libarrow to install Apache Arrow C++.

That is exactly what we're trying to install here wrt conda (and it's been working for the past four months on the same Containerfile).  mamba is just just the conda installer shipped in Jupyter.  Are you suggesting I should submit a bug about this with conda forge instead of this project?

Installing via conda gets the same error as via pip:
      --   No package 'arrow-python' found
      CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find ArrowPython (missing: ARROW_PYTHON_INCLUDE_DIR
        ARROW_PYTHON_LIB_DIR) (found version "12.0.1")
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
        cmake_modules/FindArrowPython.cmake:76 (find_package_handle_standard_args)
        CMakeLists.txt:231 (find_package)

      -- Configuring incomplete, errors occurred!
      See also "/tmp/pip-install-re2qkyh5/pyarrow_d94b37a86fab4d5883549be9791b492c/build/temp.linux-x86_64-cpython-311/CMakeFiles/CMakeOutput.log".
      error: command '/usr/bin/cmake' failed with exit code 1
      [end of output]

https://anaconda.org/conda-forge/pyarrow is a binary package.
So building pyarrow from source is strange. Are you really using https://anaconda.org/conda-forge/pyarrow ?

That's a very good point.  Let me do some digging.

Looks like this error is actually happening when installing gluonts, which was bringing in pyarrow.  Installing pyarrow standalone seems to work just fine with either pip or conda, but it seems installing gluonts from pip tries to build pyarrow even if pyarrow already exists where installing it through conda-forge is content to use the existing binary.
More curious, the build this morning pulled the wheel directly:
Collecting pyarrow~=8.0
 Downloading pyarrow-8.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (29.4 MB)
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 29.4/29.4 MB 289.0 MB/s eta 0:00:00

Where the builds as of this afternoon started pulling pyarrow in as a tar.gz.  Seems like perhaps something has happened in PyPi land today and now the source is being downloaded instead of the binary?

I just noticed that gluonts is pulling in an older version (8.0) where standalone is pulling in a newer one (12.0).  pip install pyarrow~=8.0 appears to be the problem and I think I see why.  Jupyter is now on Python 3.11 and the newest binaries for 8.0 are for 3.10.  I guess the story here is that conda-forge has a 3.11 compatible binary for its gluonts package and PyPi does not, and pyarrow 8.0 appears to be incompatible with Python 3.11.

OK. I close this.

Thanks for the thorough investigation on your end @vwbusguy !


File previews don't work when PowerToys is installed on a drive other than C:(?)

Microsoft PowerToys version
0.67.1
Installation method
WinGet
Running as admin
Yes
Area(s) with issue?
File Explorer: Preview Pane
Steps to reproduce
Not sure if what I wrote in the title is actually the case, but it has something to do with the installation's path.
With PowerToys installed on my second (E) drive – file previews in Preview Pane don't work. file-explorer-log_2023-02-28.txt
After seeing the log I tried reinstalling it onto my main (C) drive – file previews worked correctly.
I have since installed it back on my E drive – file previews are broken as before.
✔ Expected Behavior
No response
❌ Actual Behavior
Preview Pane is blank, no loading bar or any text whatsoever.

Other Software
No response

@stefansjfw I recently got a mail stating the same problem. Apparently the registry keys for the previewers always point to the c drive.
/bugreport

PowerToysReport_2023-03-01-08-07-17.zip


Database Migrations inconsistent info about sequential type

Hello
This only applies to migration type sequential.
After this commit adding type hints to system migration class. The supplied version is parsed as int.
If I want to migrate to version 002 it will parse this as int and therefore remove the 00. The class will then check for migration file with the name 2.php and not 002.php. Resulting in an error, migration file not found.
This conflicts with the old sequential style and the docs.

I tried to change my app from using 002.php to 2.php to comply with the new type hinting.
Then I get problems with MigrationRunner::findMigrations. This method checks the folder and validate the filename with regex /^\d{3}_(\w+)$/. The regex require three digits.
I'll suggest removing the int type hinting on MigrationRunner::version and MigrationRunner::checkMigrations.

@Martin-4Spaces Can you test if PR 1585 addresses your issue?

@natanfelles @jim-parry It works perfectly!


basic_hard_tip.DAE and basic_hard_tip.STL are messed up

basic_hard_tip.DAE and basic_hard_tip.STL are completely messed up on the bottom of the mesh, where there should be an opening. It looks as if some mesh optimizer was run on all the meshes, but tripped up most badly on this particular one. In general, all the meshes appear to be messy, likely for the same reason. I'm currently fixing the hard tip file, I'll try to post it here once I'm done.

I got the official model files for the tip. I have converted one to Blender format, so you can edit and re-export it if need be, STL and DAE (Collada). I noticed your models use 1 unit = 1 meter scale, the STL I generated uses common 3D-printing scale, i.e. 1 unit = 1 mm scale. Just in case I'm also providing the "downscaled" 1unit=1meter version in DAE format.
algomorph.com/storage/GripperTipOfficial.blend
algomorph.com/storage/GripperTipOfficial.dae
algomorph.com/storage/GripperTipOfficial.stl
algomorph.com/storage/GripperTipOfficial_small.dae

Really interesting! Thank you so much for fixing these meshes. We typically use them for visualization in RViz rather than for 3D printing, but having a valid mesh is always helpful. How were you able to determine that the mesh was invalid?

No problem, glad to help. A colleague of mine opened it up in MeshLab, I did the same in Blender, we both found the same problems. We have the actual gripper & tip on hand, so it was easy to tell the difference. It seems like converters from STEP and IGS CAD formats to mesh files, like STL and DAE, sometimes do a messy job, so that might have been the reason.


Geotiff visualization creates strange artifacts and does not handle correctly NaN

Hi,
I created a series of collections where it is possible to visualise the geotiff but I'm experiencing strange behaviour in certain cases.
If you look here, for example, you can find a geotiff that is correctly displayed.
This geotiff has -999 as no data.
In other cases, my geotiff has NaN as no data value. In this case, the geotiff creates the artefacts as in this example
I tried using gdal to substitute the NaN with -999 with this command:
gdal_calc.py -A $tiff --outfile=$output_folder$output_tif_nan --calc="A*(A>0)" --NoDataValue=-999 --overwrite
and with gdalinfo it results that the image has -999 as no data but if I try using this new image it results in a black image like here
What is strange is that if I open the image with QGIS it is clear and the values are correct.
Could you please tell me if I'm doing something wrong or if it is a possible bug in the browser?
The images are all stored in an Amazon S3 bucket

Hi @DanielJDufour please find here the result of the gdalinfo cmd
gdalinfo /mnt/CEPH_PROJECTS/4DMED/sen_et/4DMED_TIFs/upload_setnull_to_S3/test/Subset_SenET_Ebro_20210904.tif
Driver: GTiff/GeoTIFF
Files: /mnt/CEPH_PROJECTS/4DMED/sen_et/4DMED_TIFs/upload_setnull_to_S3/test/Subset_SenET_Ebro_20210904.tif
/mnt/CEPH_PROJECTS/4DMED/sen_et/4DMED_TIFs/upload_setnull_to_S3/test/Subset_SenET_Ebro_20210904.tif.aux.xml
Size is 8333, 4131
Coordinate System is:
GEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
CS[ellipsoidal,2],
AXIS["geodetic latitude (Lat)",north,
ORDER[1],
ANGLEUNIT["degree",0.0174532925199433]],
AXIS["geodetic longitude (Lon)",east,
ORDER[2],
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (-5.467733333000001,43.353633332999998)
Pixel Size = (0.000900000000001,-0.000900000000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  -5.4677333,  43.3536333) (  5d28' 3.84"W, 43d21'13.08"N)
Lower Left  (  -5.4677333,  39.6357333) (  5d28' 3.84"W, 39d38' 8.64"N)
Upper Right (   2.0319667,  43.3536333) (  2d 1'55.08"E, 43d21'13.08"N)
Lower Right (   2.0319667,  39.6357333) (  2d 1'55.08"E, 39d38' 8.64"N)
Center      (  -1.7178833,  41.4946833) (  1d43' 4.38"W, 41d29'40.86"N)
Band 1 Block=8333x1 Type=Float32, ColorInterp=Gray
Min=-0.000 Max=14.271
Minimum=-0.000, Maximum=14.271, Mean=3.578, StdDev=0.986
NoData Value=-999
Metadata:
STATISTICS_APPROXIMATE=YES
STATISTICS_MAXIMUM=14.271381378174
STATISTICS_MEAN=3.5783093426146
STATISTICS_MINIMUM=-0
STATISTICS_STDDEV=0.98571644098685
STATISTICS_VALID_PERCENT=32.61
thanks

@sylar11 Did you check whether the latest version on the master branch handles your file correctly?


Adds long description to setup.

Read the long description into setup so that it updates on pypi.
This will not work under python2 which is fine since 2 is EOL.
Based on https://packaging.python.org/guides/making-a-pypi-friendly-readme/

🚀


[Bug] Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.

Describe the Bug
TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
at Object.makeVolumeMetadata (chunk-63TKIGFN.js?v=9f5c3c92:33336:11)
at getVolumeInfo (@cornerstonejs_streaming-image-volume-loader.js?v=9f5c3c92:30:44)
at @cornerstonejs_streaming-image-volume-loader.js?v=9f5c3c92:1045:44
at Array.map ()
at get4DVolumeInfo (@cornerstonejs_streaming-image-volume-loader.js?v=9f5c3c92:1045:25)
at cornerstoneStreamingDynamicImageVolumeLoader (@cornerstonejs_streaming-image-volume-loader.js?v=9f5c3c92:1052:23)
at loadVolumeFromVolumeLoader (chunk-63TKIGFN.js?v=9f5c3c92:30412:28)
at Object.createAndCacheVolume (chunk-63TKIGFN.js?v=9f5c3c92:30446:22)
at VueComponent.init (View3D.vue:128:1)
Steps to Reproduce
imageids  [wadouri:http://localhost:9527/dicom/2023-11-22_91c…05.189250948959013.230502005834.4.28848.60508.dcm]
i use vite to proxy
The current behavior
TypeError: Cannot destructure property 'pixelRepresentation' of 'metaData_exports.get(...)' as it is undefined.
The expected behavior
The expected behavior
OS
Windows10
Node version
18.14.0
Browser
chrome 120.0.6099.110

Can you provide a mini repo or reproducible example? Your steps to reproduce does not have enough details

Hi.
I'm getting the very same error when using the 2.0 beta version (from the Vite repository) to create a Volume from a list of WADO-URIs image IDs.

too soon, too soon :D

Ah, just to confirm: this is in fact a bug and is not yet implemented?

Also experiencing this with the 2.0.0 versions on Next.js, occurs whenever I use WADO-URIs and showing Ortho. AXIAL view. Ortho sagittal, Ortho coronal, and Volume 3d views work fine. I detailed my issue in the issue:
https://github.com/cornerstonejs/cornerstone3D/issues/1469 (closed as a duplicate of this issue)

🥳
WORK-AROUND:
I was able to get around my issues here by programmatically re-ordering the image ids I submit to createAndCacheVolume!!!
Why:
I saw that the volume loader for wadouri: pre-fetches three slices to pull the metadata in order to build an idea of the volume before loading all slices. Often, to save space, dicom writers only include all metadata in the first slice. For my implementation, I found that the image-ids I was supplying were actually in the opposite ordering, where the final slice had all of the good metadata.
My fix:
For my specific case, I simply reversed the imageIds list and it all worked! This way, the volume loader can pre-fetch and get all the metadata it needs. This is because, when I uploaded my slices, I uploaded them in ascending order, and when I fetched them, I fetched them from my S3 folder, which returned them in descending order! I will improve my quick-fix by making sure my S3 folder fetcher returns by default, in ascending order based on datetime uploaded.
In future:
I think ideally we can figure out how to implement some fallback protection here in the wadouri: loading system:

some default or assumed values can be used if missing from the pre-fetch slices
when failing to get all required metadata, more slices are pulled at either random or best guesses (like the ending slice)

This may be the same issue as #889?

This usually happens when there are two copies of Cornerstone3D. Can you check the latest OHIF viewer, which now includes Cornerstone 3D 2.0?
Here are the migration guides:
Try OHIF locally: https://viewer-dev.ohif.org/localbasicTry Cornerstone3D 2.0 locally: https://www.cornerstonejs.org/live-examples/local.html
https://docs.ohif.org/migration-guide/3p8-to-3p9/


请问一下

May I ask whether the version of OpenLayers updated the log?

Not sure what you are asking. Please try to rephrase your question.


New plugin for the n98-magerun Magento command line tool

This is a new completion plugin for the n98-magerun tool for Magento.  https://github.com/netz98/n98-magerun

@andrew-dwyer Works on Ubuntu 14.04 on my machine.

+1

@andrew-dwyer were you able to fix @byymster's issue?

Hi @byymster, I've included your fix. Apparently some implementations of awk don't support \s.
@cmuench @mwr @mattias Could you please test the latest fix? I've also added file completion

+1 !!

+1

+1

+1

+1

+1

/cc @robbyrussell

+1

+1

+1

+1

+1

+1


[Cineverse] Unusual characters break download request

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
United States
Provide a description that is worded well enough to be understood
I discovered an issue in the Cineverse extractor where certain characters in a URL can break the extractor, as seen personally with colons in quite a few of Cineverse's titles.
The command line targeted the URL https://www.cineverse.com/details/DMR00008462/Kimagure-Orange-Road:-I-Want-to-Return-to-That-Day
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', '--skip-download', 'https://www.cineverse.com/details/DMR00008462/Kimagure-Orange-Road:-I-Want-to-Return-to-That-Day']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [f10589e34] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg n6.1.1-1-g61b88b4dda-20240120 (setts), ffprobe n6.1.1-1-g61b88b4dda-20240120
[debug] Optional libraries: Cryptodome-3.19.1, brotli-1.1.0, certifi-2023.11.17, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.1.0, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1798 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[CineverseDetails] Extracting URL: https://www.cineverse.com/details/DMR00008462/Kimagure-Orange-Road:-I-Want-to-Return-to-That-Day
[CineverseDetails] DMR00008462: Downloading webpage
[Cineverse] Extracting URL: https://www.cineverse.com/watch/DMR00008462/Kimagure Orange Road: I Want to Return to That Day
[Cineverse] DMR00008462: Downloading webpage
ERROR: DMR00008462: An extractor error has occurred. (caused by KeyError('url')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 718, in extract
  File "yt_dlp\extractor\cineverse.py", line 80, in _real_extract
KeyError: 'url'

cc @garret1317

for me it looks like the idetails is {'err_code': 1002, 'err_msg': "You don't have access to this title"}, despite as you say, everything still being there on the site
I don't have a solution at the moment, but as a workaround, downloading the same episode from retrocrush.tv works: https://www.retrocrush.tv/watch/DMR00008462/Kimagure-Orange-Road:-I-Want-to-Return-to-That-Day

If I pass cookies --cookies-from-browser firefox with an account, it starts working
and I get the same error without cookies for https://www.cineverse.com/details/DMR00018764/My-King (randomly selected one that requires login)
so I think the answer is just that they're starting to crack down on accessing stuff without proper auth, and the colons thing was just a coincidence

for me it looks like the idetails is {'err_code': 1002, 'err_msg': "You don't have access to this title"}

yeah, this is true of the watch page (https://www.cineverse.com/watch/DMR00008462/Kimagure Orange Road: I Want to Return to That Day)

it looks like the m3u8 url is available from the details page html?

what i meant by this is that the details page (https://www.cineverse.com/details/DMR00008462/Kimagure-Orange-Road:-I-Want-to-Return-to-That-Day) contains the m3u8 url even when logged out
json fragment:
"media_hls_url":"https://retrocrush-cineverse.cdn-ak.matchpoint.tv/cinedigm/DMR00008462/62c7308cf1039d14ae278756/hls/master.m3u8","media_mp4_url":"https://cdn.matchpoint.tv/lqy9glmsn5z1srgmxycy3ahe0ou7jtn2n427wtsidlb780yb5dy3ur0un7p3dio3pbc3lrtwy6cc4ummdelpjaqzul8upyj363mz95e9toszzdhu7gu3llq3tblom/4300.mp4","media":[{"url":"https://retrocrush-cineverse.cdn-ak.matchpoint.tv/cinedigm/DMR00008462/62c7308cf1039d14ae278756/hls/master.m3u8","url_clean":"https://retrocrush-cineverse.cdn-ak.matchpoint.tv/cinedigm/DMR00008462/62c7308cf1039d14ae278756/hls/master.m3u8","bitrate":"4300","duration":4265636,"type":"video/vnd.apple.mpegURL","lang":null}]


resin devices supported command does not return the list of supported device types

The docs state the resin devices supported command should return the list of device types but it doesn't. I am unsure how to find out the proper type for a RPi3
➜  database resin devices supported
Usage: resin [COMMAND] [OPTIONS]

Run the following command to get a device started with Resin.io

  $ resin quickstart

If you need help, or just want to say hi, don't hesitate in reaching out at:

  GitHub: https://github.com/resin-io/resin-cli/issues/new
  Gitter: https://gitter.im/resin-io/chat

Primary commands:

    help [command...]                   show help
    quickstart [name]                   getting started with resin.io
    login                               login to resin.io
    app create <name>                   create an application
    apps                                list all applications
    app <name>                          list a single application
    devices                             list all devices
    device <uuid>                       list a single device
    logs <uuid>                         show device logs
    sync [uuid]                         (beta) sync your changes to a device
    ssh [uuid]                          (beta) get a shell into the running app container of a device

Run `resin help --verbose` to list additional commands

Global Options:

    --help, -h
➜  database

Hi @josephroberts ,
The device supported command was removed a year ago (https://github.com/resin-io/resin-cli/pull/160), and its not being listed in the command list.
Maybe worth pinging the docs maintainer to update the CLI reference?

Hey @jviotti ,
Yes we could ping @lekkas and @shaunmulligan . The docs are out of date both at https://github.com/resin-io/resin-cli/blob/master/doc/cli.markdown and at https://docs.resin.io/tools/cli/#app-create-60-name-62-
Are they linked somehow?
Why did you state "The command is not necessary and unused."?
It seems like a nice way to figure out what the correct device type should be and directly from the CLI without having to look elsewhere. I did search for a few mins and couldn't find an obvious place where the device types are listed so its probably the same for other users.

Ah, its mentioned in the app create help command. I must have missed
that when removing it. I'll update now.

Why did you state "The command is not necessary and unused."?  It
seems like a nice way to figure out what the correct device type
should be and directly from the CLI without having to look elsewhere.
I did search for a few mins and couldn't find an obvious place where
the device types are listed so its probably the same for other users.

Yeah, maybe we should bring it back. The CLI used to have tons of
"mini-commands" back then, and I remember we had a talk about removing
the ones that no-one needed (this command was brought up there).
Happy to add back if we all agree its useful now.
On Thu, Sep 22, 2016 at 10:21:31AM -0700, Joseph Roberts wrote:

Hey @jviotti ,
Yes we could ping @lekkas and @shaunmulligan . The docs are out of date both at https://github.com/resin-io/resin-cli/blob/master/doc/cli.markdown and at https://docs.resin.io/tools/cli/#app-create-60-name-62-
Are they linked somehow?
Why did you state "The command is not necessary and unused."?
It seems like a nice way to figure out what the correct device type should be and directly from the CLI without having to look elsewhere. I did search for a few mins and couldn't find an obvious place where the device types are listed so its probably the same for other users.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/resin-io/resin-cli/issues/394#issuecomment-248969388

--
Juan Cruz Viotti
Software Engineer

I'd recommend adding it back, if it was removed. The first issue for me with the cli is always figuring out how our devices types are named (since they are non-obvious, and often non-consistent, see pi1/2/3). I remember talking about it a few months ago. I guess I just settled in with the few types that I use and remember that, but indeed, it's troublesome to try figuring out what the name of a type is that I haven't used before. Should be all accessible in the same place, i.e. within the CLI somehow. Either as a command, or listing it in the help (hard-coded, and updated as device types are updated)

Yeah it needs adding back in IMO. See below:
➜  Git resin help app create
Usage: app create <name>

Use this command to create a new resin.io application.

You can specify the application type with the `--type` option.
Otherwise, an interactive dropdown will be shown for you to select from.

You can see a list of supported device types with

	$ resin devices supported

Examples:

	$ resin app create MyApp
	$ resin app create MyApp --type raspberry-pi

Options:

    --type, -t <type>                   application type
➜  Git resin devices supported
Usage: resin [COMMAND] [OPTIONS]

Run the following command to get a device started with Resin.io

  $ resin quickstart

If you need help, or just want to say hi, don't hesitate in reaching out at:

  GitHub: https://github.com/resin-io/resin-cli/issues/new
  Gitter: https://gitter.im/resin-io/chat

Primary commands:

    help [command...]                   show help
    quickstart [name]                   getting started with resin.io
    login                               login to resin.io
    app create <name>                   create an application
    apps                                list all applications
    app <name>                          list a single application
    devices                             list all devices
    device <uuid>                       list a single device
    logs <uuid>                         show device logs
    sync [uuid]                         (beta) sync your changes to a device
    ssh [uuid]                          (beta) get a shell into the running app container of a device

Run `resin help --verbose` to list additional commands

Global Options:

    --help, -h
➜  Git

I brought back the devices supported command, and published v5.1.0 to NPM.


BFD: Data Ratios: DME Claims

Requested Feature
RIF export ratios need adjustment:

ratio of types of claims overall
ratio of claim types per bene
ratio of lines per claim type

Claim Ratio = Claim of Type / Claim Total across all types
Synthea Total:  1,054,589,695
Actual Total:  23,195,755,891

Table
Claim % (Synthea)
Claim % (Actual)
Notes

dme
0.12
2.19
slightly underrepresented in Synthea

Table
Average Line/Claim (Synthea)
Average Line/Claim (Actual)
Notes

dme
1.00
1.78

"Actual" counts as of analysis time.

Table
Count

beneficiaries
83,178,011

dme_claims
508,634,611

dme_claim_lines
906,349,376

Full data analysis for all tables is here: https://github.com/synthetichealth/synthea/wiki/RIF-Data-Analysis

Story that is emerging:

DME claims as a percentage of total claims is too low
But the probability of DME for each beneficiary is far too high
Which means that each beneficiary with a DME claim, should have more claims
DME claims should have more line items

Blood glucose was lost and found... PR #1177


Data attribute 'received' element formatted incorrectly

From what I can tell in this thread, the data attribute's 'received' element is formatted incorrectly.
Here's a sample from one of my query_sensors:
received: 2022-05-20T11:16:07+0000
That's UTC time, but missing the UTC to local offset suffix. In my case, since my time is currently EDT, I think it's supposed to look like this to allow for conversion to local time:
received: 2022-05-20T11:16:07-04:00
I assume that if this were configured properly, the -04:00 would automatically change to -05:00 when daylight savings time ends.
I've confirmed my time zone is properly set in O365.

Which attribute of which sensor is this?

Oops, you said

OK, looked at it further. The date is formatted correctly for what can be provided by the integration because the integration is backend processing, it would need something at the front-end to format the dates for local user time. What you see in Developer Tools is not (in the most part) local timezone aware. I say in the most part, because the start and end time at the root of the calendar entities are formatted by HA at the front end and do show in local time, so do not show +/-xxxx
The +0000 denotes that is UTC.
So a date that says 2022-05-20T11:16:07+0000 is the same as 2022-05-20T12:16:07+0100 or 2022-05-20 12:16:07 in British Summer Time which is GMT+1.
Now of course if you are saying the mail was received at 11:16:07 local time and your are in EDT then it should show 15:16:07+0000, which when you translate to local time would be correct to your eyes.

Just to add, the backend (where the integration runs), does not know your local timezone so cannot format for it. It may coincidently be that HA is in the same timezone, that cannot be relied on. Any front-end timezone is configured against each user and the backend does not know which user is accessing the data, and therefore what timezone the user is viewing in.
Recently HA has taken the decision that all backend dates will be handled in UTC and only presented in local time by the front-end.

The message was received at 7:16 AM EDT (11:16 UTC).
I agree those three British Summer Times would be equivalent, but without the offset in the data, I cannot easily account for DST in templates (without changing the offset twice a year)...AFAIK.
Do you know of a way to access the GMT offset in the frontend? If I could use that as a variable, I think I'd be set.

I’m afraid not, I’ve not got engaged in HA fronted at all. You could always hard code it, but of course you then need to account for when DST charges for you. There is probably a restapi driven service out there as well that you could pull down the relevant time zone info with a rest sensor, or even one you could push the date/time to that would convert it for you.

OK, thanks. I guess another reason why having this field included as a query sensor variable would be better since it would be handled on the backend.

Please see my comment on your other issue since I need to understand your use case. I’ll close this one.

Just thinking out loud...
After comparing/contrasting the three exampls ways to represent the same time at the beginning of this comment, I think something is still off considering:
{{ now() }}
gives me
2022-05-20 12:37:00.029958-04:00
Which is my local time EDT (UTC-4). So, it's providing me with a local time and the UTC offset which is not a combination that post suggests is correct (either the time should be 16:37 or the offset should be dropped). I'll post the question in that post.

https://community.home-assistant.io/t/the-epic-time-conversion-and-manipulation-thread/85786/242?

Yep, because you are doing that in the front-end, not in the backend. Reality is, I'm not changing how the data is presented, you may process it how you like ;-)

Answer here...just a mistake on his part. The UTC representation is the local time appended with the UTC offset...not the UTC time appended with the UTC offset

You are right and wrong. When people say 'UTC' they mean +0000 (i.e. the time at London Greenwich timeline ignoring DST). There are different ways of representing the same time as I showed above, but they all mean the same thing. For HA it has been requested that dates are shown at +0000. You may if you wish translate that to local time in the front-end, since you know how far distant you are from UTC, whilst the backend doesn't.

Yeah, it's definitely splitting hairs at this point, but his post is about proper formatting, so I was just trying to clarify. I'm with you though...I know what time zone I'm in and can do math ;-)

Looking some more (I tend to do that). The requirement from HA is that dates are represented in UTC time zone (which arguably not a thing). To represent in UTC, to be more correct it should be formatted with a ‘Z’ instead of ‘+0000’. However with 1k plus users I’m not inclined to change it when they mean the same thing.

https://www.w3.org/TR/NOTE-datetime

Looks like you may be able to use this format to present the date in local time in Lovelace.
{{ states.calendar.bookings.attributes.data[0].start|as_timestamp|timestamp_local }}
No idea if you can use that to create a template sensor, but certainly works in templates tab of developer tools.


Add get_name to Seedable

Calling pure virtual methods in the constructor is dangerous. The problem here is that Seedable calls add_callback_function, and in turn it calls
REQUIRE(
	    has_parameter(tag), "There is no parameter called \"%s\" in %s\n",
	    tag.name().c_str(), get_name());

Here get_name is pure virtual, and it's not yet defined because the chain of construction didn't reach the derived class yet. The only reason this doesn't segfault, or produce the error pure virtual method called terminate called without an active exception is the lazy nature of macros.

What would be the output of this error message now with this PR?

get_name would return "Seedable" instead of the derived class name. But that's the case with any other class if construction failed midway. A better name would be "SeedableParent::get_name()", but that won't work when Parent is SGObject.

Mmmh this is pretty annoying.... but OK at least these errors will only ever occur for devs


Devel docs remarks

Here's some things I noted in the current docs on the devel branch. Some of these are more general than specifically for 2.0, but I figured I'd list them anyway.
Incorrect:

The bgColor parameter is listed as valid for all renderers, but the path tracer doesn't listen to it, see https://github.com/ospray/ospray/issues/347
For isosurfaces geometry and volume texture the volume parameter needs to reference an OSPVolumetricModel apparently, although the docs say it's of type OSPVolume. For example https://github.com/ospray/ospray/blob/764afe7d094bd09324739dd678e27543b4d949f8/apps/tutorials/ospTutorialStructuredVolume.cpp#L96 This is probably due to the need for accessing the transfer function for geometry coloring?
The signature for ospNewFrameBuffer is out of date w.r.t. ospray.h, as it uses a osp_vec2i instead of two ints

Unclear/ambiguous:

The type of the volume and geometry parameters of an OSPGroup is listed as OSPData, so I would assume ospSetData is the right call to use, but the tutorials use ospSetObject. For example: https://github.com/ospray/ospray/blob/764afe7d094bd09324739dd678e27543b4d949f8/apps/tutorials/ospTutorialStructuredVolume.cpp#L155
How interchangeable are those two calls anyway?

Is there a way to color an isosurface by setting a material on the
GeometricModel? I.e. bypassing the TF completely for cases where one isn't needed?

What does the path tracer's geometryLights parameter do exactly?

The exact memory layout and semantics of an affine3f is specified nowhere, nor does ospray.h give any clues

Is a subdivision geometry the go-to mesh structure when you don't have a pure-tri or pure-quad? Would setting level to 0 (i.e. no subdivision) give similar performance as the pure-tri/quad meshes or would it be better to convert an n-gon mesh into tris?

Is ospCommit needed on an OSPData when it has just been created with ospNewData? From https://github.com/ospray/ospray/blob/8d5ff6d1e197f24e4ab6966ef2c5d82cbee60412/apps/tutorials/ospTutorial.c#L103-L106 it seems yes, but from https://github.com/ospray/ospray/blob/8d5ff6d1e197f24e4ab6966ef2c5d82cbee60412/apps/tutorials/ospTutorialStructuredVolume.cpp#L32-L34 it seems no.

Is the reference count of an OSPObject influenced by ospCommit? Or is it immediately increased when doing ospSetObject/ospSetData The latter is suggested by https://github.com/ospray/ospray/blob/8d5ff6d1e197f24e4ab6966ef2c5d82cbee60412/apps/tutorials/ospTutorial.c#L113-L118, but there's also https://github.com/ospray/ospray/blob/8d5ff6d1e197f24e4ab6966ef2c5d82cbee60412/apps/tutorials/ospTutorial.c#L126-L131

Good to know:

Color and opacity arrays given to ospNewTransferFunction do not have to have the same length
When using imageStart and imageEnd on an OSPCamera the resulting image will show the requested cropped region of the camera frustrum, but using the full framebuffer resolution for the cropped region. This is different from having the imageStart/imageEnd specify a region in the original framebuffer before cropping (resulting in a cropped region that is a subset of the framebuffer).
When not using the OSP_DATA_SHARED_BUFFER flag on ospNewData a copy of the source array is made. Also, if an array of OSPObject's is passed (e.g. OSPLight) all the objects in the array will have their reference count incremented.

An spheres geometry apparently wants a sphere data array for the positions, but is listed as spheres in the docs.

ospSetObject and ospSetData are actually aliases.

https://github.com/ospray/ospray/blob/9527f27984f6ba7daeb3d64df7c0d4615866ddab/README.md#L741-L746
But an error is sent to the error handler in this case:
null handle provided to void ospRelease(OSPObject)

Is there a way to color an isosurface by setting a material on the GeometricModel? I.e. bypassing the TF completely for cases where one isn't needed?

Yes, an isosurface is a geometry like all others and can get a material assigned (via the geometric model). However, a TF is currently still required for the volumetric model (we may relax this in future, in particular for this use case).

Is a subdivision geometry the go-to mesh structure when you don't have a pure-tri or pure-quad? Would setting level to 0 (i.e. no subdivision) give similar performance as the pure-tri/quad meshes or would it be better to convert an n-gon mesh into tris?

Mixed tri/quad meshes can be handled by the "mesh" geometry, higher n-gons should better be tessellated, because the subdivision geometry is rather costly.


⚠ Bofrost.at has degraded performance

In 59224e3, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1319 ms

Resolved: Bofrost.at performance has improved in fd7da1f.


Need to write post about new ressource on nodejs.org/learn

We just get this new ressource on learn section. and we need to post something about it
https://nodejs.org/en/learn/modules/publishing-a-package

@JakobJingleheimer Do you want to post it yourself and have us repost it or just post it (and mention you)?

Yeah, I shall do 😊

Trying to publish a package and there are just. so. many. options? The good news is it's probably not as complicated as it seems. We gotchu. Check out the new https://nodejs.org/en/learn/modules/publishing-a-package

"there are just. so. many. options?" have little bit to much dot why not use comma ?

Also Jacob you don't want to be mentioned as writer ? Maybe we can reply/create a thread to say that you are the author

BTW we need to create a second JSON file for the replying because the actual post engine didn't support thread feature

"there are just. so. many. options?" have little bit to much dot why not use comma ?

This is a dramatisation. I'll send you an audio message for how it reads.

Also Jacob you don't want to be mentioned as writer ? Maybe we can reply/create a thread to say that you are the author

Ah, sure. If it's posted by the nodejs account 🙂 if i post it, that would be weird 😹
If so, just append by @jakob.jingleheimer.dev

i let you open the pr. If you didn't have the time just say it

I don't know how to make the json. If you don't mind doing it, please do (and tag me so i can see how it's done) 🙏


How to add america/chicago timezone time to jquery bootstrap timepicker?

I used jquery bootstrap timepicker.It takes only local time.But in that timepicker i want to show to america/chicago timezone value.So how can i get this?

Duplication of #803?


Enable element-wise weighing of outputs in model.fit(..., sample_weight=weights)

Hi everyone,
I'd like to enable model.fit(..., sample_weight=weight) to support not only 1D vectors (one element per training input), or 2D matrices for temporal series, but n-D arrays of the same size as the output, so that we can weigh the loss element-wise, e.g. instead of
w [scalar] * Loss(y_true, y_pred) [3D array]

we can have
W [3D array] * Loss(y_true, y_pred) [3D array]

I think the basic functionality is already there, and training doesn't need to be changed. For example, weighted_masked_objective(fn) in training_utils.py contains the code
        score_array = fn(y_true, y_pred)
...
        # apply sample weighting
        if weights is not None:
            # reduce score_array to same ndim as weight array
            ndim = K.ndim(score_array)
            weight_ndim = K.ndim(weights)
            score_array = K.mean(score_array,
                                 axis=list(range(weight_ndim, ndim)))
            score_array *= weights
            score_array /= K.mean(K.cast(K.not_equal(weights, 0), K.floatx()))

as if weights is the same size as score_array, then *= would do element-wise multiplication, at least with TensorFlow as the backend. Instead, all we need is some small changes in model.compile() to accept n-D arrays for weights, and allocate placeholders accordingly.
I've forked keras and written a short script to start testing this (with tensorflow-gpu-1.8.0)
import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
import keras
import keras.backend as K
import numpy as np

from keras.models import Sequential
from keras.layers import Activation, Conv2D
from keras.layers.normalization import BatchNormalization

# remove warning "Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'  # Just disables the warning, doesn't enable AVX/FMA

K.set_image_data_format('channels_first')

# simulate input images
im = np.zeros(shape=(10, 3, 64, 64), dtype='uint8')

# simulate network output
out = 2 * np.ones(shape=(10, 1, 64, 64), dtype='float32')

# simulate training weights for network output
weight = np.ones(shape=(10, 1, 64, 64), dtype='float32')
#weight = np.zeros(shape=(10, ), dtype='float32')

# create network model
model = Sequential()
model.add(Conv2D(input_shape=(3, 64, 64),
                 filters=32, kernel_size=(3, 3), strides=1, padding='same'))
model.add(BatchNormalization(axis=3))
model.add(Activation('relu'))
model.add(Conv2D(filters=1, kernel_size=(1, 1), strides=1, padding='same'))

optimizer = keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)

model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'], sample_weight_mode='element')

model.fit(im, out, sample_weight=weight, batch_size=3, epochs=3)

The first change I suggest is adding a new option sample_weight_mode='element' to model.compile(). This has to be added to compile() in training.py in a couple of places, to cover different number of outputs and weight formats (dict, list, array) of sample_weight_mode. For example, for the case of a single output and weights given as an array, we'd add to this if block in training.py
                    if sample_weight_mode == 'temporal':
                        sample_weights.append(
                            K.placeholder(ndim=2,
                                          name=name + '_sample_weights'))
                        sample_weight_modes.append('temporal')
                    else:
                        sample_weights.append(
                            K.placeholder(ndim=1,
                                          name=name + '_sample_weights'))
                        sample_weight_modes.append(None)

this snippet
                    if sample_weight_mode == 'temporal':
                        ...
                    elif sample_weight_mode == 'element':
                        output_shape = self.get_output_shape_at(0)
                        sample_weights.append(
                            K.placeholder(ndim=len(output_shape)-1,
                                          name=name + '_sample_weights'))
                        sample_weight_modes.append('element')
                    else:
                        ...

At this point, the test script above runs successfully
/home/rcasero/.conda/envs/cytometer_tensorflow/bin/python /snap/pycharm-community/64/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client <IP_ADDRESS> --port 46211 --file /home/rcasero/Software/cytometer/scripts/keras_branch_sample_weight.py
pydev debugger: process 19509 is connecting

Connected to pydev debugger (build 181.5087.37)
Using TensorFlow backend.
Backend TkAgg is interactive backend. Turning interactive mode on.
Epoch 1/3
Python 3.6.3 |Anaconda, Inc.| (default, Nov  3 2017, 19:19:16)
[GCC 7.2.0] on linux
[2.0, 1.0]

 3/10 [========>.....................] - ETA: 33s - loss: 1.9810 - acc: 1.0000
10/10 [==============================] - 14s 1s/step - loss: 1.9442 - acc: 1.0000
Epoch 2/3

 3/10 [========>.....................] - ETA: 0s - loss: 1.8317 - acc: 1.0000
10/10 [==============================] - 0s 1ms/step - loss: 1.7663 - acc: 1.0000
Epoch 3/3

 3/10 [========>.....................] - ETA: 0s - loss: 1.5962 - acc: 1.0000
10/10 [==============================] - 0s 1ms/step - loss: 1.5120 - acc: 1.0000

Process finished with exit code 0

+1

New sample_weight_mode='element' was breaking when training with validation_data (validation_split worked fine). Fixed by creating a validation weights array of 1.0s with the same size as val_y in model.fit().
                if self.sample_weight_mode == 'element':
                    val_sample_weight = np.ones(shape=val_y.shape, dtype=float)
                else:
                    val_sample_weight = None

A test script for this is
import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
import keras
import keras.backend as K
import numpy as np

from keras.models import Sequential
from keras.layers import Activation, Conv2D
from keras.layers.normalization import BatchNormalization

# remove warning "Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'  # Just disables the warning, doesn't enable AVX/FMA

K.set_image_data_format('channels_first')

# simulate input images
im = np.zeros(shape=(10, 3, 64, 64), dtype='uint8')

# simulate network output
out = 2 * np.ones(shape=(10, 1, 64, 64), dtype='float32')

# simulate training weights for network output
weight = np.ones(shape=(10, 1, 64, 64), dtype='float32')

# simulate validation data
im_validation = 3 * np.ones(shape=(5, 3, 64, 64), dtype='uint8')
out_validation = 4 * np.ones(shape=(5, 1, 64, 64), dtype='float32')
validation_data = (im_validation, out_validation)

# create network model
model = Sequential()
model.add(Conv2D(input_shape=(3, 64, 64),
                 filters=32, kernel_size=(3, 3), strides=1, padding='same'))
model.add(BatchNormalization(axis=3))
model.add(Activation('relu'))
model.add(Conv2D(filters=1, kernel_size=(1, 1), strides=1, padding='same'))

optimizer = keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)

model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'], sample_weight_mode='element')

# train with validation_split
#model.fit(im, out, sample_weight=weight, validation_split=0.1, batch_size=3, epochs=3)

# train with validation_data
model.fit(im, out, sample_weight=weight, validation_data=validation_data, batch_size=3, epochs=3)

Keras API Design Review google doc with comments enabled here
https://docs.google.com/document/d/19BDXgNmeTgpgb9xYKzNboXyM7XX2PeM3mlvCFCdIQj0/edit?usp=sharing

This issue in the keras user mailing list
https://groups.google.com/forum/#!topic/keras-users/IgQYtlSYCp0

Pull request created here
https://github.com/keras-team/keras/pull/10775
as suggested by David Menéndez Hurtado.

At this point, some tests fail in the pull request automatic testing. I have asked for guidance in the thread "How to debug a keras pull request" in the mailing list.

New testing script:
import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
import keras
import keras.backend as K
import numpy as np

from keras.models import Model, Sequential
from keras.layers import Activation, Conv2D, Input
from keras.layers.normalization import BatchNormalization

from keras.utils import multi_gpu_model

# remove warning "Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'  # Just disables the warning, doesn't enable AVX/FMA

# image_data_format = 'channels_first'
image_data_format = 'channels_last'

K.set_image_data_format(image_data_format)

# simulate input images
if image_data_format == 'channels_first':
    im = np.zeros(shape=(10, 3, 64, 64), dtype='uint8')

    # simulate network output
    out = 2 * np.ones(shape=(10, 1, 64, 64), dtype='float32')
    aux_out = 5 * np.ones(shape=(10, 1, 22, 22), dtype='float32')
    # simulate training weights for network output
    weight = np.ones(shape=(10, 1, 64, 64), dtype='float32')
    aux_weight = np.ones(shape=(10, 1, 22, 22), dtype='float32')

    # simulate validation data
    im_validation = 3 * np.ones(shape=(5, 3, 64, 64), dtype='uint8')
    out_validation = 4 * np.ones(shape=(5, 1, 64, 64), dtype='float32')

elif image_data_format == 'channels_last':
    im = np.zeros(shape=(10, 64, 64, 3), dtype='uint8')

    # simulate network output
    out = 2 * np.ones(shape=(10, 64, 64, 1), dtype='float32')
    aux_out = 5 * np.ones(shape=(10, 22, 22, 1), dtype='float32')
    # simulate training weights for network output
    # weight = np.ones(shape=(10, 64, 64, 1), dtype='float32')
    weight = np.ones(shape=(10, 64, 64, 1), dtype='float32')
    aux_weight = np.ones(shape=(10, 22, 22, 1), dtype='float32')

    # simulate validation data
    im_validation = 3 * np.ones(shape=(5, 64, 64, 3), dtype='uint8')
    out_validation = 4 * np.ones(shape=(5, 64, 64, 1), dtype='float32')

else:
    raise ValueError('Unrecognised position for channels')

validation_data = (im_validation, out_validation)

# optimizer
optimizer = keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)

'''simple CNN with one output
'''

# create network model
model = Sequential()
model.add(Conv2D(input_shape=im.shape[1:],
                 filters=32, kernel_size=(3, 3), strides=1, padding='same'))
model.add(BatchNormalization(axis=3))
model.add(Activation('relu'))
model.add(Conv2D(filters=1, kernel_size=(1, 1), strides=1, padding='same'))

'''string format (sample_weights_mode='element')
'''

# compile model
model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'], sample_weight_mode='element')

# train with validation_data
model.fit(im, out, sample_weight=weight, validation_data=validation_data, batch_size=3, epochs=3)

'''dictionary format (sample_weight_mode={'conv2d_2': 'element'})
'''

model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'], sample_weight_mode={'conv2d_2': 'element'})

# train with validation_data
model.fit(im, out, sample_weight=weight, validation_data=validation_data, batch_size=3, epochs=3)

'''parallel training in multiple GPUs
   string format (sample_weights_mode='element')
'''

# list all CPUs and GPUs
device_list = K.get_session().list_devices()

# number of GPUs
gpu_number = np.count_nonzero(['GPU' in str(x) for x in device_list])

if gpu_number > 1:  # compile and train model: Multiple GPUs

    # compile model
    parallel_model = multi_gpu_model(model, gpus=gpu_number)
    parallel_model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'], sample_weight_mode='element')

    # train model
    parallel_model.fit(im, out, sample_weight=weight, validation_data=validation_data, batch_size=gpu_number, epochs=3)

else:
    raise Warning('Cannot test parallel GPU processing, because I could not find 2 or more GPUs')

'''Multi-output CNN
'''

# create network model
input = Input(shape=im.shape[1:], dtype='float32')
x = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding='same')(input)
x = BatchNormalization(axis=3)(x)
x = Activation('relu')(x)

main_output = Conv2D(filters=1, kernel_size=(1, 1), strides=1, padding='same', name='main_output')(x)
aux_output = Conv2D(filters=1, kernel_size=(1, 1), strides=3, padding='same', name='aux_output')(x)

model = Model(inputs=input, outputs=[main_output, aux_output])

'''list format (sample_weight_mode=['element', 'element'])
'''

model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'],
              sample_weight_mode=['element', 'element'])

model.fit(im, [out, aux_out], sample_weight=[weight, aux_weight],
          batch_size=3, epochs=3)

'''dictionary format (sample_weight_mode={'conv2d_2': 'element'})
'''

model.compile(loss='mae', optimizer=optimizer, metrics=['accuracy'],
              sample_weight_mode={'main_output': 'element', 'aux_output': 'element'})

model.fit(im, {'main_output': out, 'aux_output': aux_out},
          sample_weight={'main_output': weight, 'aux_output': aux_weight},
          batch_size=3, epochs=3)

Currently all Travis builds pass.


ARCHBOM-1047: fix make upgrade for django-user-tasks

Made the following updates:

new constraints

django-model-utils<4.0.0
djangorestframework<3.10.0

drop Python 2.7 with major version bump to 1.0.0

python 3 quality fixes

dependency clean-up

use base.in to load requirements from setup.py
move minimum constraints from setup.py to base.in
move celery pin to constraints.txt
moved pip-tools to a separate file
started to compile and reference base.txt

ARCHBOM-1047

@timmc-edx: I missed the CHANGELOG.  I'll add that before merging.


Reading from json in Cucumber scenario outline

QAF Version

Note: 2.1.14

I have started exploring Qaf bdd framework lately and found it amazing. I got one issue when using Cucumber Scenario outline with QAF, I have given path to external json file as test data. But while executing @QAFTestStepProvider not able to read integer from json. It reading string just fine. Please help me here. Is it a bug or there is a way around for this
This is my json test data
[
{ "searchKey" : "Rupak", "searchResult":"Testng", "number": 96 },
{ "searchKey" : "Admin", "searchResult":"Yuont", "number": 78 },
{ "searchKey" : "Qa", "searchResult":"Umbrella", "number": 121 }
]

Can you share your scenario and step implementation as well?
This is not right place for questions or discussion. Please post your questions and/or discussions in qaf users group or stackoverflow with qaf tag.


Can't run 'ansible-playbook main.yml' on Pi itself

I have done everything else I am
pi@monitor-pihole:~/internet-monitoring $ ansible --version
ansible 2.9.23

but every time I go to run the play book I get the below issues.
First time I tried:
pi@monitor-pihole:~/internet-monitoring $ ansible-playbook main.yml

PLAY [internet_pi] ****************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************
The authenticity of host '<IP_ADDRESS> (<IP_ADDRESS>)' can't be established.
ECDSA key fingerprint is SHA256:T6yUqbdRb9Q6DYv7vDeH6zMc0M2qU06r8XadjvpuIzU.
Are you sure you want to continue connecting (yes/no)? yes
fatal: [<IP_ADDRESS>]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '<IP_ADDRESS>' (ECDSA) to the list of known<EMAIL_ADDRESS>Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP ************************************************************************************************************************************************
<IP_ADDRESS>              : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

pi@monitor-pihole:~/internet-monitoring $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet <IP_ADDRESS>  netmask <IP_ADDRESS>  broadcast <IP_ADDRESS>`

2nd time I have tried to run it I got the below
`pi@monitor-pihole:~/internet-monitoring $ ansible-playbook main.yml

PLAY [internet_pi] ****************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************
fatal: [<IP_ADDRESS>]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh<EMAIL_ADDRESS>Permission denied (publickey,password).", "unreachable": true}

PLAY RECAP ************************************************************************************************************************************************
<IP_ADDRESS>              : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0 `

Are you trying to run the playbook on the Pi itself? If so there are a couple things you need to do slightly differently (and I can guide you through them... maybe get that added to the README too!)
Otherwise, make sure you can SSH into the Pi (ssh<EMAIL_ADDRESS>if you're running from another computer.

Correct I am ssh in tot he pi right now and I downloaded all the file from your repository to my PI I just built.
pi@monitor-pihole:~/internet-monitoring $ ls -la
total 68
drwxr-xr-x 6 pi pi 4096 Jun 23 17:55 .
drwxr-xr-x 8 pi pi 4096 Jun 23 18:01 ..
-rw-r--r-- 1 pi pi   86 Jun 23 17:18 ansible.cfg
-rw-r--r-- 1 pi pi  487 Jun 23 17:55 config.yml
-rw-r--r-- 1 pi pi  504 Jun 23 17:18 example.config.yml
-rw-r--r-- 1 pi pi  159 Jun 23 17:18 example.inventory.ini
drwxr-xr-x 3 pi pi 4096 Jun 23 17:47 .github
-rw-r--r-- 1 pi pi   25 Jun 23 17:18 .gitignore
drwxr-xr-x 2 pi pi 4096 Jun 23 17:47 images
-rw-r--r-- 1 pi pi  161 Jun 23 17:51 inventory.ini
-rw-r--r-- 1 pi pi 1080 Jun 23 17:18 LICENSE
-rw-r--r-- 1 pi pi  855 Jun 23 17:18 main.yml
-rw-r--r-- 1 pi pi 2875 Jun 23 17:18 README.md
-rw-r--r-- 1 pi pi   38 Jun 23 17:18 requirements.yml
drwxr-xr-x 2 pi pi 4096 Jun 23 17:52 tasks
drwxr-xr-x 2 pi pi 4096 Jun 23 17:47 templates
-rw-r--r-- 1 pi pi  107 Jun 23 17:18 .yamllint
I have a Pi that I was just using for PiHole so I decided I would do your play book to just run it all on the Pi4 I have with 8 Gb ram So this pi use to do just pihole but I wipped it and said he I will just have it handing off my router and run your setup so I can just have it all in one place on the pi itself.

I am also just running this as pi
Should I sudo and run the commands?

Okay, so in this case, what you'd want to do is the following:

In inventory.ini, change the line under [internet_pi] to read: localhost ansible_connection=local
Try running main.yml again.

Hopefully that will work!

I am getting this right now.

@clumsyzombie - Before running the playbook, try deleting the directory: rm -rf /home/pi/internet-monitoring. Looks like it may have been cloned already?

So I did that and I am not able to access any web service's still on the pi after running it so I am not sure anything been copied over yet if I am getting errors.
So by deleting that I am not able to issue ansible-playbook main.yml   because that is in the same directory as everything so when you issue ansible-playbook main.yml   does it copy everything to another place? Could it be that something is not copy over.

So I did resolve one error the error above was unable to resolve so I got DNS working now and I can resolve google.com and github.com now
So I am getting this error now

@clumsyzombie - It looks like there are still DNS issues there; because Docker is trying to do a docker pull on the image but is not able to see the registry. It seems like it's trying to query DNS on the Pi itself, but check out https://github.com/geerlingguy/internet-pi/issues/8 as well.

@geerlingguy You might have a laugh about this I went into /etc/resolvconf.conf  it already was not committed so I said what the heck let me commit it out. Everything ran because I have DNS setup already on the pie else where. I did that and it fully installed.
Now I am in Grafana and looking around and pihole is up and running I need to config it but I can access each.

Excellent!

im having the same issue what do i do

nvm i figured it out

@geerlingguy

@clumsyzombie - Beautiful! For the Speedtest Graph it will still take some time before it shows a more stable graph over a long period (and looks nicest at 6 hours or more zoomed out).

It looks like the main issue's been resolved at this point; the other issue that can happen on first run is https://github.com/geerlingguy/internet-pi/issues/22 — so moving the rest of this discussion there.

I am getting this error when I run “ansible-playbook main.yml:
pi@RPi4-PLEX:~/internet-pi $ ansible-playbook main.yml
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
The error appears to have been in '/home/pi/internet-pi/main.yml': line 15, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
handlers:
- name: Include handlers.
^ here
Ive ssh’ed into my pi4 and everything seems good setup wise. The inventory.ini file contains the following:
[internet_pi]
<IP_ADDRESS> ansible_user=pi
Comment out the previous line and uncomment this to run inside Raspberry Pi.
<IP_ADDRESS> ansible_connection=local ansible_user=pi
Nothing in main.yml seems to indicate a path.
Thanks in advance,
Nb

@netbob - Please make sure you're running Ansible 2.9 or later (ansible --version to check).

@netbob - Please make sure you're running Ansible 2.9 or later (ansible --version to check).

Thanks! That worked so all runs well until it hits this portion:
TASK [Ensure {{ ansible_user }} user is added to the docker group.] ****************fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible_user' is undefined\n\nThe error appears to be in '/home/pi/internet-pi/tasks/docker.yml': line 44, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Ensure {{ ansible_user }} user is added to the docker group.\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - "{{ foo }}"\n"}
PLAY RECAP *************************************************************************localhost                  : ok=7    changed=2    unreachable=0    failed=1    skipped=2    rescued=0    ignored=0
pi@RPi4-PLEX:~/internet-pi $

@netbob - Make sure your inventory file has the whole contents of the line that was fixed a few days back: https://github.com/geerlingguy/internet-pi/blob/master/example.inventory.ini#L5 (including the ansible_user=pi part).

Adding the ansible_user=pi line worked for me. It works great, thanks for the assist Jeff.

@geerlingguyBạn có thể cười về điều này. Tôi đã vào /etc/resolvconf.conf nó chưa được cam kết nên tôi đã nói cái quái gì vậy, hãy để tôi cam kết nó. Mọi thứ đều chạy vì tôi đã thiết lập DNS ở chỗ khác. Tôi đã làm điều đó và nó đã được cài đặt đầy đủ.
Bây giờ tôi đang ở Grafana và nhìn xung quanh và thấy pihole đang hoạt động. Tôi cần định cấu hình nó nhưng tôi có thể truy cập từng cái.

Where did you go to /etc/resolvconf.conf file?

localhost ansible_connection=local

It would save many persons time I imagine to add this point of note to the readme of the git.


CDK construct can't change and deploy a resolver if changed from @function to custom appsync JS resolver

Amplify CLI Version
12.2.5
Question
Following on from this issue  #3003  - I can now deploy multiple GSI changes iteratively with the CDK construct. Thanks to @AnilMaktala and @sundersc
The only issue now that seems to cause problems for multiple devs deploying their changes is if a resolver has been created with the @function and attached a lambda, then a dev changes this in the schema and replaces with a custom appsync JS resolver, we get the following error:
12:57:01 PM | CREATE_FAILED
Resource already exists: arn:aws:appsync:us-west....
HandlerErrorCode: AlreadyExists)
In order to deploy we have to remove the @function code, deploy the Appsync api stack, then add our new appsync resolver to the schema and then deploy again.
This is a problem for the committing to a pipeline that deploys live as we'd need to deploy twice to switch changes live?

Hey 👋 , This issue is being closed due to inactivity. If you are still experiencing the same problem and need further assistance, please feel free to leave a comment. This will enable us to reopen the issue and provide you with the necessary support.


Indicating time trends in topN results

This issue elaborates on one of the aspects mentioned in the discussion about Error bounds / probabilities / skewness as first-class Druid query results: time trends for single-valued query types such as topN and groupBy. In different time intervals, results may be different, for example:
interval=00:00_01:00
country    website_visits
China      2000
Argentina  1000
USA        900

...
interval=05:00_06:00
country    website_visits
Argentina  1500
USA        1300
China      1000

-- China goes to bed, Americas wake up.
When we aggregate just those 5 hours, there is a relative trend for each dimension value (country). We may detect this when aggregating results on Broker and send a enum along with the value for each dimension value or grouping key:

consistent upward trend
consistent downward trend
no trend (the relative weight of the key stays realtively stable), or a cycling trend: relative weight goes up and down and finishes approximately at the same place where it has started.
big variance, but no particular trend.

Then in query UIs, consistent upward and downward trends may be represented as small  green and red arrows (those that are used in stock market interfaces :)
Big variance, but no particular trend may be indicated by something like an asterisk  or different tone of the row or an exclamation mark in a circle, indicating that the total aggregate value probably has low statistical significance, or there is some problem with the data (see #7160).
FYI @mistercrunch @julianhyde @vogievetsky

I don't know what algorithm is being used to establish the "TopN", but I hope it isn't the old

Sort each node
Take the top 1000 from each node
Merge the samples from each node together and sort
Take the top 100 from that list and hope (yes hope) that you have captured the top 100.

This can result in unknown and significant errors, depending on how the data is distributed across the nodes, and cannot give any information about which ones in that final list actually belong there, what the error might be in its position in the list, and which ones in the list might just be noise.  This technique is just a heuristic and it is not hard to prove how it can return very misleading results.
This can be can be much more efficiently, accurately, and confidently solved using the DataSketches Frequent Items sketch (FIS).
The FIS will tell you the upper and lower bounds of the actual weights of each item, which can be used to establish the range within which the true weight must lie.  However, when those ranges overlap, it creates a tie situation in terms of actual order.  Nonetheless, these bounds can be used to identify which items may be tied, for example, for the 3rd position.  These bounds, by the way are hard bounds, so the confidence is 100%.
Also, you can query the sketch using either the NO_FALSE_POSITIVES (more strict)  or a NO_FALSE_NEGATIVES (less strict) query.  But under no circumstances with the sketch return items that fall below the noise floor.
Be aware that the reason that this sketch is called "Frequent Items" is that there may not be any items that are more frequent than any of the others (I.e. if they all occur with the same frequency).  Thus, returning zero results can happen. Thus the name "TopN" is misleading as there may not be N items returned.
This is why I prefer to call this kind of operation "Frequent Items" or "Heavy Hitters" and not "TopN".

@leerho yes, we do exactly the "old" algorithm that you described :) See #1135. Thank you for letting us know about how the algorithm can be improved! I've opened a separate issue #7187 to track your proposal.

@leventov
BTW, I have checked into DataSketches/sketches-core a new sketch I have specifically developed with this issue in mind for Druid.  It is called Frequent Distinct Tuples (FDT Sketch.  I would appreciate it if someone could experiment with it.  So now there are 2 sketches we have to help Druid deal with “TopN” issues. This plus the FIS Sketch.  I can make myself available for a short tutorial if anyone is interested on how to get started.

@leerho I support your stance in #7187. But, unfortunately, I won't be able to work myself on these ideas in the near future.

@leerho Do you have any documentation on what the FDT sketch is capable of? I'm curious how it can be mapped onto two important things: use cases and SQL.

@gianm
Yes, the initial docs are the Javadocs, (duplicated on the website) I am working on more extensive docs including some accuracy characterization studies, which are more complicated to do.  Meanwhile:  See FdtSketch Javadoc.
Also see unit tests for some real simple examples to get started.  But don't use the FdtSketch(lgK) to configure, use the FdtSketch(threshold, rse) to configure the sketch. You can start out by trying numbers like FdtSketch(0.01, 0.05), which means you will capture distincts that are greater than at least 10% of the overall population of distincts where the Relative Standard Error (at 68% confidence) will be 5% or better.

@gianm
see #7187


Metdata(EXIF,TIFF,GPS etc..) stripped off from the saved temp image

Version Number of Plugin: 2.6.2.
Device Tested On: Iphone 6 IOS 10.0
Simulator Tested On: IOS Simulator
Version of VS: VS 2015
Version of Xamarin: 4.5
When we try to save the Image after picking it from the Camera Roll using PickPhotoAsync(), the photo is created in the temp directory of the device but the Metadata is stripped off during that save, which should not happen, it should maintain all of the metadata of the original image.
To fix it I replaced the following line of code in the MediaPickerDelegate.cs for the Media.Plugin.IOS project:
Method Name : SaveImageWithMetadata
I replaced :
destination.AddImageAndMetadata(cgImageFromJpeg, cgImageMetadata, destinationOptions);
	            var success = destination.Close();
	            if (success)
	            {
	                imageWithExif.Save(path, true);
	            }
                return success;`

with :

              var imgSrc = ImageIO.CGImageSource.FromData(image.AsJPEG());
             var outImageData = new NSMutableData();
            using (var d = ImageIO.CGImageDestination.Create(outImageData, imgSrc.TypeIdentifier, 1, new ImageIO.CGImageDestinationOptions()))
            {
                d.AddImage(imgSrc, Convert.ToInt32(imgSrc.ImageCount - 1), meta);
                d.Close();
            }
            NSError writeError;
            var success = outImageData.Save(path, NSDataWritingOptions.Atomic, out writeError);

            return success;

It did work but I still wanted to know why the previous code the you had was not working.
thanks

These are in the 2.7.0 beta


Add Seeed XIAO RP2040 board

New Seeed board:
https://www.seeedstudio.com/XIAO-RP2040-v1-0-p-5026.html
Not sure about this line:
#define PICO_BOOT_STAGE2_CHOOSE_GENERIC_03H 1

It comes with a GD25Q16CEIG - 2MB SPI NAND Flash.
Not sure if that is compatible with W25Qxx used on other boards, and if it should be changed to:
#define PICO_BOOT_STAGE2_CHOOSE_W25Q080 1

It's kinda awkward that you have to go to the wiki page and then look at the schematics to work out what RP2040 pin numbers they've actually used for different things! :confused:  But AFAICT the numbers you've used here look correct :+1:

it sounds like this is not ready to merge, so i have converted it to a draft

GENERIC_03H and W25Q080 are both working for me on the seeed board.
The flash is actually puya P25Q16H.
Related: https://github.com/adafruit/circuitpython/pull/5519

@mcauser PICO_DEFAULT_LED_PIN should be 25 (the schematic is outdated or something) and you should set PICO_DEFAULT_LED_PIN_INVERTED to 1

@mcauser see #691 - I'll leave it up to you and @ryanplusplus to work out what you want to do :slightly_smiling_face:

@mcauser see #691 - I'll leave it up to you and @ryanplusplus to work out what you want to do :slightly_smiling_face:

Sounds like #691 supersedes this PR, so I guess this PR should be closed? :shrug:

closing - reopen if need be


fix payments link (bug 1154264)

I trust that this URL is correct. r+ but wait to see if tests pass.


help

please update/tell me where the offsets are located because i cant find them anywhere :(i have searched the whole thing

Look at Entity.h vtable idx changes. The Offsets are no problem they were signature proofed if you know what I mean.

first of all nice work for the pull request'
second you should post ur problems in the issues tab
third this cheat uses signature scan instead of offsets so there are no offsets in this cheat.


RemoveResponseHeader filter not removing all the set-cookie headers

I have a below route configuration to remove set-cookie headers from the response header. RemoveResponseHeaderfilter is not removing all the set-cookie header, I see still few set-cookie headers in the response.

id: xxx-xxx-service
uri: lb://xxx-xxx-service
predicates:
- Path=/api/yyy/**
filters:
- StripPrefix=2
- RemoveResponseHeader=set-cookie

It is not able to remove the below set-cookie headers.
set-cookie: ADRUM_BTa=R%3A0%7Cg%3Af803a77f-1f3b-4abb-ad98-b2af77da363f%7Cn%3Acustomer1_9d14899b-9f1f-428a-a5d5-9094eb3d2365; Max-Age=30; Path=%2F
set-cookie: ADRUM_BT1=R%3A0%7Ci%3A935172%7Cd%3A141; Max-Age=30; Path=%2F
set-cookie: SameSite=None; Max-Age=30; Path=%2F; Secure
But it is able to remove the below set-cookie headers.
Set-Cookie: ADRUM_BTa=R:0|g:9b04eddf-88b0-4040-babd-6c4193f0b842; Max-Age=30; Expires=Wed, 24-Aug-2022 08:21:23 GMT; Path=/; Secure
Set-Cookie: ADRUM_BTa=R:0|g:9b04eddf-88b0-4040-babd-6c4193f0b842|n:customer1_9d14899b-9f1f-428a-a5d5-9094eb3d2365; Max-Age=30; Expires=Wed, 24-Aug-2022 08:21:23 GMT; Path=/; Secure
Set-Cookie: SameSite=None; Max-Age=30; Expires=Wed, 24-Aug-2022 08:21:23 GMT; Path=/; Secure
one thing what I observed is the set-cookie headers which it is not able remove have path attribute encoded.
Please let us know am I doing any mistake here in the configuration, is there any other way to remove all the set-cookie header.

Are all those coming from the downstream response? Are any of those generated by gateway?

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

Hi engineers,
The same problem appeared.

Can you also try set Max-Age = 0 to delete cookie?


Feature/add page viewcounts

Initial setup managing article viewcounts.

Creates view_counts table
Adds path to articles to make google analytic queries easier
Tidies up belongs_to/has_many relations on existing models

ping @edenspiekermann/backend


Decoding to fluid objects appears to fail in recent versions of SBCL

Decoding JSON to fluid objects appears to not work in recent versions of SBCL. For example, the following works fine in SBCL 1.1.0 on Linux (Ubuntu 14.04.3):
* (format t "~@{~A~%~}"
        (asdf:component-version (asdf:find-system :cl-json))
        (lisp-implementation-type)
        (lisp-implementation-version)
        (software-type)
        (software-version)
        (machine-type)
        (machine-version))
0.5.0
SBCL
1.1.0
Linux
3.16.0-50-generic
X86-64
Intel(R) Core(TM) i7-2760QM CPU @ 2.40GHz
NIL
* (json:with-decoder-simple-clos-semantics
    (json:decode-json-from-string
     "{ \"foo\": [1, 2, 3], \"bar\": true }"))

#<#<JSON:FLUID-CLASS NIL {10055708B3}> {100560D693}>

However, in the latest SBCL, 1.3.2, it fails:
* (format t "~@{~A~%~}"
        (asdf:component-version (asdf:find-system :cl-json))
        (lisp-implementation-type)
        (lisp-implementation-version)
        (software-type)
        (software-version)
        (machine-type)
        (machine-version))
0.5.0
SBCL
1.3.2
Linux
3.16.0-50-generic
X86-64
Intel(R) Core(TM) i7-2760QM CPU @ 2.40GHz
NIL
* (json:with-decoder-simple-clos-semantics
    (json:decode-json-from-string
     "{ \"foo\": [1, 2, 3], \"bar\": true }"))

debugger invoked on a SB-PCL::SLOTD-INITIALIZATION-ERROR in thread
#<THREAD "main thread" RUNNING {1002A4C6E3}>:
  Invalid SB-MOP:SLOT-DEFINITION initialization: the initialization argument
  :NAME was constant: :FOO.
See also:
  AMOP, Initialization of Slot Definition Metaobjects

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

((:METHOD INITIALIZE-INSTANCE :BEFORE (SB-MOP:SLOT-DEFINITION)) #<SB-MOP:STANDARD-DIRECT-SLOT-DEFINITION {10049862D3}> :NAME :FOO :INITFORM #<unused argument> :INITFUNCTION #<unused argument> :TYPE #<unused argument> :ALLOCATION NIL :INITARGS NIL :DOCUMENTATION NIL) [fast-method]
0]

It appears to have begun failing in this way at SBCL version 1.2.0 or earlier:
* (format t "~@{~A~%~}"
        (asdf:component-version (asdf:find-system :cl-json))
        (lisp-implementation-type)
        (lisp-implementation-version)
        (software-type)
        (software-version)
        (machine-type)
        (machine-version))
0.5.0
SBCL
1.2.0
Linux
3.16.0-50-generic
X86-64
Intel(R) Core(TM) i7-2760QM CPU @ 2.40GHz
NIL
* (json:with-decoder-simple-clos-semantics
    (json:decode-json-from-string
     "{ \"foo\": [1, 2, 3], \"bar\": true }"))

debugger invoked on a SB-PCL::SLOTD-INITIALIZATION-ERROR in thread
#<THREAD "main thread" RUNNING {1002BF5053}>:
  Invalid SB-MOP:SLOT-DEFINITION initialization: the initialization argument
  :NAME was constant: :FOO.
See also:
  AMOP, Initialization of Slot Definition Metaobjects

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

((:METHOD INITIALIZE-INSTANCE :BEFORE (SB-MOP:SLOT-DEFINITION)) #<SB-MOP:STANDARD-DIRECT-SLOT-DEFINITION #<unbound slot>> :NAME :FOO :INITFORM #<unused argument> :INITFUNCTION #<unused argument> :TYPE #<unused argument> :ALLOCATION NIL :INITARGS NIL :DOCUMENTATION NIL) [fast-method]
0]

I tried to poke at this, but went astray when trying to inspect the slot definition. It seems to have no name, which either borks the SBCL inspector, or the SLIME interface to the inspector.

Thanks for reporting, I have quite little time to look at this now sadly.

Seems to be a quirk of sbcl. It doesn't like it when you use keyword package for MOP stuff. As a workaround, if you change cl-json:json-symbols-package to your own package, it works okay.

It also doesn't work  on LispWorks (and never did), because LispWorks also does not allow constant symbols as slot names.


sudo apt-get udpate commnad error with respect of ansible

root@ip-172-31-43-138:~# apt-get update
Err:1 http://ppa.launchpad.net/ansible/ansible/ubuntu xenial InRelease
Could not connect to ppa.launchpad.net:80 (<IP_ADDRESS>), connection timed out
0% [Connecting to us-east-1.ec2.archive.ubuntu.com (<IP_ADDRESS>)] [Connecting^                                          C

seems to be packet failed .
Did you followed https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#latest-releases-via-apt-ubuntu ?


0.7.1 release

[ ] #67

Let's give it one more day of testing & look to release late tomorrow

CHANGELOG is updated.


Roll engine 033f2072c315..163a2fd5e140 (26 commits)

https://github.com/flutter/engine/compare/033f2072c315...163a2fd5e140
git log 033f2072c315d48b4d7680d4cbbe856012b1259b..163a2fd5e140d10a30645e73031054ef39105d87 --no-merges --oneline
163a2fd5e Revert "Android embedding refactor pr3 add remaining systemchannels (#7738)" (flutter/engine#7849)
0d6ff1669 Shut down and restart the Dart VM as needed. (flutter/engine#7832)
256db4bc2 Android embedding refactor pr3 add remaining systemchannels (flutter/engine#7738)
02d7ca31d Don't call static method from instance variable (flutter/engine#7841)
10cee612b Delete GL textures when they are released from the texture registry. (flutter/engine#7836)
69e460607 Fix NullPointerException in SurfaceTextureRegistryEntry (flutter/engine#7837)
9a965bc3d Fix NullPointerException in ResourceCleaner (flutter/engine#7838)
2acd79480 Add fml::FileExists implementation for Windows (flutter/engine#7845)
fa38a985c Update buildroot to 7f64ff4928e to unblock Mac builds. (flutter/engine#7846)
1ba329550 Add support for calling into other plugins from a background context on iOS (flutter/engine#7843)
df2fc973a Roll src/third_party/skia 154acd7a1374..be9aff25bddc (13 commits) (flutter/engine#7842)
0ca1d1f31 Remove the Dart JIT snapshot data from AOT builds of the embedder library (flutter/engine#7806)
c51ea4157 Roll src/third_party/skia 7a74c7cb6da0..154acd7a1374 (1 commits) (flutter/engine#7840)
ecbaea10a Ensure to pass dill file after VM options for gen_snapshot (flutter/engine#7839)
5d3f7145a Roll src/third_party/skia b00f7b34751b..7a74c7cb6da0 (1 commits) (flutter/engine#7835)
733739938 Roll src/third_party/skia 9a88bee122f1..b00f7b34751b (24 commits) (flutter/engine#7831)
6f318efe0 Move up ndk version that is being downloaded(old one no longer available) (flutter/engine#7830)
a8aa1ee06 Track flow id (flutter/engine#7826)
b1ce6b72f Roll src/third_party/skia a0dcd29f536b..9a88bee122f1 (4 commits) (flutter/engine#7825)
dbc166387 Roll src/third_party/skia 6152470dc69e..a0dcd29f536b (2 commits) (flutter/engine#7824)
942677695 Roll src/third_party/skia 4037f7f5d8b4..6152470dc69e (1 commits) (flutter/engine#7823)
e2394ad77 Add flow events connecting pointer events to frames (flutter/engine#7807)
0a5a7c56d Roll src/third_party/skia 09c01e9df260..4037f7f5d8b4 (3 commits) (flutter/engine#7821)
b867f4858 Roll src/third_party/skia 186669c4128b..09c01e9df260 (5 commits) (flutter/engine#7820)
af39e22c3 Fix tests that were committed after cirrus ran (flutter/engine#7819)
1f2cbf792 Roll src/third_party/dart 0a7dcf17eb..c92d5ca288 (64 commits)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

Auto-roller completed checks. About to merge.

@cbracken fyi this had a measurable impact on footprint.

Traced this to an increase in the size of libflutter.so due to the addition of the Dart FFI package


Is it working for "enum" and "class"?

Hi @hipstersmoothie ,
I just tried your lib' but when generating the docs it brings no additional additional for enum and class definitions. I'm just seeing:
UtmParams (type)
UtmSynapse (class)

Maybe I did something wrong... was it working for you?
Thank you,
EDIT: I switched to https://typedoc.org/ at the end

@sneko I experienced result similar to yours. I'm only showing class names but nothing regarding anything of their properties or methods.
I'm looking for a solution that embeds the documentation into the README.md. Can https://typedoc.org/ do that? I was not able to find anything on it. Thanks!


VS Code on Windows 7 - NextGen ActionScript extension exited with error code 1

I am trying to set up the nextgenas extension for VS Code.  I must be doing something wrong because, I am getting this error:

I am not sure if there are any more detailed logs I can dig in.  If there are any, please let me know where I can find them.

This is usually caused by multiple versions of Java installed on your computer. 0.1.0 picks the first Java installation it finds, and if that's not Java 8, it fails with an error and gives up. I've fixed that for 0.2.0, and it will continue to search for more versions of Java after a failure. 0.2.0 needs more time, so I'm thinking that I need to do a 0.1.1 release because, as you can probably guess, you aren't the first to have this issue.

Sure, that would be great.  If there is a branch for 0.1.1, I can grab it and test it before you do an actual release.  If that would be helpful for you, that is.

I emailed you a build, Om. No point in having you set up a dev environment for a quick test. Especially since I'm considering some changes to the build soon.

Did you send an email?  I never hit anything.
On Nov 4, 2016 12:57 PM, "Josh Tynjala"<EMAIL_ADDRESS>wrote:

I emailed you a build, Om. No point in having you set up a dev environment
for a quick test. Especially since I'm considering some changes to the
build soon.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/BowlerHatLLC/vscode-nextgenas/issues/16#issuecomment-258532213,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABktdWC3oVnY8jWi8bs9BYjtNudwWLY-ks5q644YgaJpZM4KpGrl
.

Yeah, I sent it to your Gmail, with the subject "vscode-nextgenas 0.1.1 build"

Okay, I saw it in the spam folder.
I uninstalled the existing one and installed it with the .vsix file you sent me.  I am still getting the same error.  Now, in addition to that I am seeing these in the OUTPUT window:
[Error - 2:23:43 PM] Connection to server is erroring. Shutting down server.
[Error - 2:23:43 PM] Connection to server is erroring. Shutting down server.
[Error - 2:23:43 PM] Connection to server is erroring. Shutting down server.
[Error - 2:23:43 PM] Connection to server is erroring. Shutting down server.
[Error - 2:23:43 PM] Connection to server is erroring. Shutting down server.
[Error - 2:23:43 PM] Connection to server is erroring. Shutting down server.

Can you press Ctrl+Shift+P and run the Developer: Toggle Developer Tools command? Can you tell me what's in the console?

I see multiple such errors in the console:
extensionHost.ts:255[Extension Host] TypeError: Cannot read property 'dispose' of null     at LanguageClient.cleanUp (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-languageclient\lib\main.js:631:30)     at LanguageClient.stop (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-languageclient\lib\main.js:605:14)     at LanguageClient.handleConnectionError (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-languageclient\lib\main.js:866:18)     at errorHandler (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-languageclient\lib\main.js:728:19)     at C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-languageclient\lib\main.js:47:42     at CallbackList.invoke (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-jsonrpc\lib\events.js:103:39)     at Emitter.fire (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-jsonrpc\lib\events.js:167:36)     at readErrorHandler (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-jsonrpc\lib\main.js:110:22)     at CallbackList.invoke (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-jsonrpc\lib\events.js:103:39)     at Emitter.fire (C:\Users\omuppirala\.vscode\extensions\bowlerhatllc.vscode-nextgenas-0.1.1\node_modules\vscode-jsonrpc\lib\events.js:167:36)

Are you running VSCode 1.7.1?
Did you restart VSCode after installing the extension?

Yes, I did restart after installing the extension

Please try the new build that I emailed to you!

Thanks for the patience on this one, Om. Can you please try installing the new version 0.3.0 of the extension in VSCode? Let me know if the error code has changed when it crashes (it might be something other than 1 now). Also, please share any errors that appear in the developer tools. Thank you!

Thanks for ping back!
I am just setting up my new Windows 10 PC.  This was one of the first
things I was going to test :-)
If you still want me to test on Windows 7, please let me know.
Thanks,
Om
On Tue, Jan 24, 2017 at 11:17 AM, Josh Tynjala<EMAIL_ADDRESS>wrote:

Thanks for the patience on this one, Om. Can you please try installing the
new version 0.3.0 of the extension in VSCode? Let me know if the error code
has changed when it crashes (it might be something other than 1 now). Also,
please share any errors that appear in the developer tools. Thank you!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/BowlerHatLLC/vscode-nextgenas/issues/16#issuecomment-274906836,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABktdep0RQSXi-sgZKEmHYFOFuXd-d31ks5rVk5lgaJpZM4KpGrl
.

Hey, Om. If everything works fine on Windows 10, I'd like you to double-check on your Windows 7 machine, just to be sure. Thanks!

Tested the new version on Windows 7.  Much better!  I dont see any errors
in the console and things seem to be working.  I need to play around with
it more to build and compile an app.
But things are looking much better already.
Thanks,
Om
On Tue, Jan 24, 2017 at 11:22 AM, Josh Tynjala<EMAIL_ADDRESS>wrote:

Hey, Om. If everything works fine on Windows 10, I'd like you to
double-check on your Windows 7 machine, just to be sure. Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/BowlerHatLLC/vscode-nextgenas/issues/16#issuecomment-274908444,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABktdefZXtC3HdQ11ROFE0IGglp_OLmmks5rVk99gaJpZM4KpGrl
.

Good news!