No TypeConverter for Color - Maybe add one

If I create a control with an AvaloniaProperty of type Color, it cannot be conveniently set from XAML
public static readonly DirectProperty<Ctl, Color> OnColorProperty =
          AvaloniaProperty.RegisterDirect<Ctl, Color>(nameof(OnColor),
          o => o.OnColor, (o, v) => o.OnColor = v, olor.Transparent,  BindingMode.Default);

This assignment
   <adb:Ctl OnColor="Brown" />
or that
   <adb:Ctl OnColor="#FF4455" />
will fail, because there is no TypeConverter for Color.
In comparison see IBrush:
    /// <summary>
    /// Describes how an area is painted.
    /// </summary>
    [TypeConverter(typeof(BrushConverter))]
    public interface IBrush

Would it be an Idea to add this to the Avalonia-Code for Color?
Are there reasons there is none? Or should I try to fix this issue? Or anybody else?

@Robertofon I think it just needs a type converter adding, feel free to open a PR, should pretty much be a copy and paste of the brush one Im guessing...
a PR will be a good way to force a discussion about it too... issues can often get lost or overlooked.

Compiler should be using Color.Parse.

I've added this PR https://github.com/AvaloniaUI/Avalonia/pull/4552
to sohow the problem I'm having.
Direct assignment thrugh xaml works as @kekekeks says. But style setters do not.
\avaloniaui.net\avalonia\src\Avalonia.Styling\Styling\Setter.cs
has the issue somhow hidden in it:
              data.result = new PropertySetterInstance<T>(
                    data.target,
                    property,
                    (T)data.value);

I dont know what to do. Anyways a TypeConverter does not fix this problem.

            Style style = new Style(x => x.OfType<Class1>())
            {
                Setters =
                {
                    new Setter(Class1.TestColorProperty, "#11223344"),
                },
            };

This code won't work since XAML compiler is not involved. You need to pass an actual Color when constructing styles from code

Seems like it fixed itself in the meantime - I did not relaize I can upgrade to 0.10 and since then it works

sorry to make effort


Added method to EntityContext to check data from stored Entities

I propose to include a new step in the EntityContext to be able to check if some record is registered in the database, matching by specified field.
In my case, we needed to check if a record or a modification is done correctly by checking the properties of the entity, something cannot be done with the current steps.
Hope you find it interesting.

Seems good...
Is there any other way to emulate an entity ? https://github.com/KnpLabs/FriendlyContexts/pull/121/files#diff-10f6572966ac1c44a5a56d8184874157R142

I don't know other way. Since PropertyAccessor is created inside the method I cannot stub it. So I had to pass a "stub" of the entity simulating a real returned entity.
Actually I just removed some wrong promises (check my second commit) that weren't doing anything.
I guess to avoid that we should inject a PropertyAccessor instance to the Context class as dependency.
Of course any suggestion is welcome.

@javierseixas this feature won't work as we need to know the identifier of object, isn't it ?
I propose to find object from record bag instead.


enumitem incompatibility with phase-III

I am using an up to date build of TeX Live (packages are now frozen to July 2023).
I cannot compile a document that loads enumitem and defines a custom list of specific types if I set testphase=phase-III in \DocumentMetadata{}
I am compiling the following MWE with lualatex --synctex=1 --interaction=nonstopmode --file-line-error enumitem.tex:
\DocumentMetadata{
 % does not always work
 testphase=phase-III,
 % always works
 %testphase=phase-II,
 pdfversion=2.0,
 pdfstandard=A-4
}

\documentclass[a4paper,10pt]{article}
\setlength{\parindent}{0mm}

\usepackage{enumitem}

% does not work with phase-III
% new enumerate with numeric label
\newlist{mylist}{enumerate}{1}
\setlist[mylist,1]{label={\bfseries ML \arabic*.}}

% does not work with phase-III
% new enumerate with static label
%\newlist{mylist}{enumerate}{1}
%\setlist[mylist,1]{label={\bfseries ML A}}

% does not work with phase-III
% new itemize with numeric label
%\newlist{mylist}{itemize}{1}
%\setlist[mylist,1]{label={\bfseries ML \arabic*.}}

% always works, also with phase-III
% new itemize with static label
%\newlist{mylist}{itemize}{1}
%\setlist[mylist,1]{label={\bfseries ML A}}

\begin{document}
Text before
\begin{mylist}
 \item Item A
 \item Item B
\end{mylist}
Text after
\end{document}

Output for phase-III:
This is LuaHBTeX, Version 1.17.0 (TeX Live 2023/Arch Linux)
 restricted system commands enabled.
(./enumitem.tex
LaTeX2e <2022-11-01> patch level 1
 L3 programming layer <2023-02-22>
(/usr/share/texmf-dist/tex/latex/latex-lab/documentmetadata-support.ltx
(/usr/share/texmf-dist/tex/latex/pdfmanagement-testphase/pdfmanagement-testphas
e.sty (/usr/share/texmf-dist/tex/latex/tagpdf/tagpdf-base.sty)
(/usr/share/texmf-dist/tex/latex/pdfmanagement-testphase/pdfmanagement-testphas
e.ltx (/usr/share/texmf-dist/tex/latex/l3experimental/l3bitset/l3bitset.sty))))
 (/usr/share/texmf-dist/tex/latex/l3backend/l3backend-luatex.def)
(/usr/share/texmf-dist/tex/latex/pdfmanagement-testphase/l3backend-testphase-lu
atex.def (/usr/share/texmf-dist/tex/latex/pdfmanagement-testphase/l3ref-tmp.sty
))
(/usr/share/texmf-dist/tex/latex-dev/latex-lab/phase-III-latex-lab-testphase.lt
x (/usr/share/texmf-dist/tex/latex/latex-lab/phase-II-latex-lab-testphase.ltx
(/usr/share/texmf-dist/tex/latex/latex-lab/latex-lab-testphase-new-or.sty
(/usr/share/texmf-dist/tex/latex/latex-lab/latex-lab-footnotes.ltx))
(/usr/share/texmf-dist/tex/latex/tagpdf/tagpdf.sty
(/usr/share/texmf-dist/tex/latex/tagpdf/tagpdf-luatex.def)
(/usr/share/texmf-dist/tex/latex/tagpdf/tagpdf-mc-code-lua.sty)))
(/usr/share/texmf-dist/tex/latex-dev/latex-lab/latex-lab-testphase-block-taggin
g.sty (/usr/share/texmf-dist/tex/latex/l3packages/xtemplate/xtemplate.sty
(/usr/share/texmf-dist/tex/latex/l3kernel/expl3.sty))
(/usr/share/texmf-dist/tex/latex-dev/latex-lab/latex-lab-kernel-changes.sty
(/usr/share/texmf-dist/tex/generic/pdftex/glyphtounicode.tex)
(/usr/share/texmf-dist/tex/latex-dev/latex-lab/glyphtounicode-cmex.tex)))
(/usr/share/texmf-dist/tex/latex-dev/latex-lab/latex-lab-testphase-sec-tagging.
sty)
(/usr/share/texmf-dist/tex/latex-dev/latex-lab/latex-lab-testphase-toc-tagging.
sty))
(/usr/share/texmf-dist/tex/latex/pdfmanagement-testphase/pdfmanagement-firstaid
.sty) (/usr/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(/usr/share/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texmf-dist/tex/latex/enumitem/enumitem.sty)
(/usr/share/texmf-dist/tex/latex/pdfmanagement-testphase/output-patches-tmp-ltx
.sty (/usr/share/texmf-dist/tex/latex/etoolbox/etoolbox.sty)
Patching header in \@outputpage
Patching footer in \@outputpage
) (./enumitem.aux) (/usr/share/texmf-dist/tex/latex/base/ts1cmr.fd)<<sRGB.icc>>

./enumitem.tex:34: Missing \endcsname inserted.
<to be read again>
\global
l.34  \item I
           tem A
./enumitem.tex:34: You can't use `\relax' after \the.
<recently read> \c@

l.34  \item I
           tem A
./enumitem.tex:34: Missing \endcsname inserted.
<to be read again>
\protect
l.34  \item I
           tem A
./enumitem.tex:34: Illegal parameter number in definition of \@currentHref.
<to be read again>
1
l.34  \item I
           tem A
./enumitem.tex:34: Illegal parameter number in definition of \@currentHref.
<to be read again>
1
l.34  \item I
           tem A
./enumitem.tex:34: Extra \or.
\enit@saveset ...l \c@enit@cnt \endcsname {#4}\or
                                                  \expandafter \let \expanda...

l.34  \item I
           tem A
./enumitem.tex:35: Missing \endcsname inserted.
<to be read again>
\global
l.35  \item I
           tem B
./enumitem.tex:35: You can't use `\relax' after \the.
<recently read> \c@

l.35  \item I
           tem B
./enumitem.tex:35: Missing \endcsname inserted.
<to be read again>
\protect
l.35  \item I
           tem B
./enumitem.tex:35: Illegal parameter number in definition of \@currentHref.
<to be read again>
1
l.35  \item I
           tem B
./enumitem.tex:35: Illegal parameter number in definition of \@currentHref.
<to be read again>
1
l.35  \item I
           tem B
./enumitem.tex:35: Extra \or.
\enit@saveset ...l \c@enit@cnt \endcsname {#4}\or
                                                  \expandafter \let \expanda...

l.35  \item I
           tem B
[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}] (./enumitem.aux))
(\end occurred when \ifcase on line 35 was incomplete)
(\end occurred when \if on line 35 was incomplete)
(\end occurred when \if on line 34 was incomplete)
(\end occurred when \if on line 34 was incomplete)
(see the transcript file for additional information)
 432 words of node memory still in use:
   3 hlist, 1 vlist, 1 rule, 2 glue, 3 kern, 2 glyph, 5 attribute, 51 glue_spec
, 5 attribute_list, 1 write nodes
   avail lists: 1:1,2:164,3:72,4:6,5:91,6:8,7:194,9:52,11:4
</usr/share/texmf-dist/fonts/opentype/public/lm/lmroman10-bold.otf></usr/share/
texmf-dist/fonts/opentype/public/lm/lmroman10-regular.otf>
Output written on enumitem.pdf (1 page, 25045 bytes).
Transcript written on enumitem.log.

phase-III loads the new list implementation and that is currently not compatible with enumitem. You will have to wait until we provide a replacement.

However, most of the extended syntax provided bybenumitem is already available out of the box with the new implementation provided by phase-III, but  the declarations to define your own list environments or configure existing ones in "enumitem"-style is not offered.

Do you have any clue on why the fourth case succeeds?

@AlphaJack probably because it is not trying to manipulate a counter in contrast to case two, but it is nothing that is worth spending time on in isolation. As Ulrike said: at a later stage we will have to look how the full interface can be offered given that the underlying list implementation has changed and what is necessary for that to be provided. But this has to wait.


Subexports and treeshaking

While discussing whether or not we should move to monorepo with @cpreston321 and @alex-eliot; I thought that maybe instead we should only improve the exports and start testing the package for treeshaking.
That would also be a great opportunity for both of you to make a tour of all package exports and maybe start thinking of some improvements that could be made to general structure.
What I would expect:

@vueuse/motion exports everything "user-facing" at component level like <Motion />, useMotion and other composables
@vueuse/motion/presets exports all the presets and related purposes
@vueuse/motion/plugin would maybe only export the plugin (could also be exposed from @vueuse/motion but maybe we can promote a "plugin-less" usage in long term?)

@Tahul; I actually like this take of making smaller improvements to exports! I think this might be a better approach. I can see what I can do by making small improvements first.


Estimated release date?

Omg! This is a great tool for the community. Is there an estimated release date for the code?

The code will be released within this month. Thanks for your interest!

mark

Very much looking forward to it, too.

The code has been released and the detailed instruction will be fully released in the next week, since I'm occupied by many other issues recently.


Remove template parameters from copy constructor

C++23 no longer allow template parameters in constructor

@microsoft-github-policy-service agree

With this change, were you able to compile LightGBM using the C++20 (or even better, C++23 standard)? Or is this just one step towards that?

Can confirm I can compile with C++23, gcc-12. However, it requires me to remove the __mm_malloc macros - https://github.com/microsoft/LightGBM/issues/4331 (but this seem be a GCC)

Ok interesting, thank you for that! I don't recall the history for these _mm_malloc macros:
https://github.com/microsoft/LightGBM/blob/3d386be29bb9e8032e9acaec7c831133928fbeba/include/LightGBM/utils/common.h#L41-L55
Maybe @StrikerRUS or @guolinke will remember why they were added in #2699.

sometimes we need the memory allocation with aligned address, like https://github.com/microsoft/LightGBM/blob/484283202be456800abc3ba57039e8028536d593/include/LightGBM/cuda/vector_cudahost.h#L41-L60

Ah got it, thank you! Ok I'll investigate this further. Maybe some headers have changed in newer versions of compilers.


Fix crashes when saving 0 as custom tag name

Details

Saving the tag name as 0 will crash the app. This PR fix it.
Fixed Issues

$ https://github.com/Expensify/App/issues/43773
PROPOSAL: https://github.com/Expensify/App/issues/43773#issuecomment-2168142274
Tests

Same as QA Steps

[x] Verify that no errors appear in the JS console

Offline tests

Same as QA Steps
QA Steps

Go to workspace tags page
Press Settings
Press the tag name
Rename the tag to 0
When it fails, verify the tag name reverts to the previous name
Press the close error button
Verify the error is cleared

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/50919443/c07a84d8-75ff-41ae-8156-c2dbb6cca8a3

Android: mWeb Chrome

https://github.com/Expensify/App/assets/50919443/01b0b77c-c3ab-4049-a499-edd62fedc302

iOS: Native

https://github.com/Expensify/App/assets/50919443/1c28cb97-60bd-4d7f-a5ca-f9acd6ed4e3e

iOS: mWeb Safari

https://github.com/Expensify/App/assets/50919443/7e1dcec6-7c78-4794-80d7-72d993608450

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/50919443/27efb84c-cb77-477b-83e5-94aa7d5fe16f

MacOS: Desktop

https://github.com/Expensify/App/assets/50919443/5537fde2-9794-4fc1-bc3f-b573d8223281

Oh, I thought it's already fixed. I will take a look at it later. Idk why somehow the network request keep stuck.
I did:

Change the name to 0
Change the name to a valid name
Change the name to 0 (stuck here, never processed)

Ok, so for the other issue, it's related to the BE too. If we update the tag name from A to B, we will optimistically merge:
A: null (old tag)
B: newTagData
then, if we quickly change the name from B to 0 (or others) before the first rename request completes, we will optimistically merge:
B: null (old tag)
0: newTagData
After the first request completes (A -> B), the BE response will merge the B tag data.

So now, we have 2 tag lists, B and 0 which is detected as multi-level tags. After the second request fails, the 0 tag is removed, so we are back with 1 tag.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/danieldoglas in version: 9.0.5-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Cherry-picked to staging by https://github.com/Julesssss in version: 9.0.5-4 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by https://github.com/Julesssss in version: 9.0.5-13 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/thienlnam in version: 9.0.6-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


FailedSOPSequence in dicom upload

Files are being stored in the storage data archive but  no data is saved  in the database. Connection string works perfectly. Response  from server is FailedSOPSequence (0008,1197)with code 272. What could i be missing.

@OdungaTM this means there is an issue with the data in your DICOM dataset (missing required elements, invalid data...) causing the insert to DB operation to fail.
Did you look at the trace logs or exception being thrown?

@OdungaTM I believe this is now fixed in "Server fails to store when a DICOM file is invalid (valu doesn't confirm to VR) #99"
Can you verify?

closed as potentially fixed in #99


Add started RPC metric for client and server side

/bin/sh: 1: goimports: not found

Seems like CI is broken

Yeah, will do once it's merged. Thanks.

test fix PR is merged.  Feel free to rebase

Done.

goimports FAILED => goimports the following files:
make: *** [Makefile:49: imports] Error 1

./plugin/ocgrpc/client_metrics.go
./plugin/ocgrpc/end_to_end_test.go
./plugin/ocgrpc/server_metrics.go

I think you might need a go fmt ./...


Test contrib smart ci

@akladiev is it still needed? may we close it?

@akladiev is it still needed? may we close it?

Yes, will close it for now


Benchmark and consider AOS instead of SOA

SOA is usually the better option for performance (citation needed)
But in an archetype based system we can expect systems utilizing archetype A  would want to utilize most components per entity. Maybe this is something users should be able to control themself, because my hunch is that it might be very case by case based
Suggested syntax:
const World = WorldBuilder().WithArchetypes(
    Archetype.A, // default is SOA
    AOS(Archetype.B), // Force B to be stored as AOS
    SOA(Archetype.C), // Explicitly use SOA storage
);

I'm closing this because the api is very different at this point. I still think AOS vs SOA is a interesting topic that should be explored in the future ...


Can frame extraction before labeling be done via command line or even sped up with DALI?

Hi, LP team:
can lightning pose extract frames by command line or even can the extracting be speeded up by using dali?
It seems that the extracting is also time consuming, especially when doing k-means clustering like DLC.

@Wulin-Tan are you referring to a DLC function or a lightning-pose/Pose-app function?
I should mention that in the Pose-app repo we have functions for extracting frames from raw videos and saving them. I initially implemented this using DALI but believe it or not I found OpenCV to be slightly faster (which is what is implemented now). Much of the time required for this operation isn't the video reading but rather running PCA+kmeans on the downsampled frames.

Hi, @themattinthehatt

exactly! I recently tried the  extracting and found that there was no advantage for dali. And the most time consuming step is the clustering for frames, so now what I am doing is to do resizing  and grayscale on cpu, and then do the clustering on gpu with python packages like skorch. This would make the preprocessing before extracting much faster(In my toy data, 10min down to 20s).  I am wondering whether there are independent functions in LP to do the preprocessing like resizing and grayscale even on gpu?
do you mean that I need to install Pose-app to get the function?

oh wow 10 min -> 20 s is an impressive speedup! I'll have to look into skorch, thanks for the pointer.
The resizing and grayscale happens on GPU with DALI, that's why it's surprising that it's not faster than OpenCV 🤷♂
You wouldn't necessarily need to install Pose-app if you just wanted to use that function, you could just copy-paste into your own pipeline - but it sounds like your skorch implementation is much faster than our opencv+sklearn function, so you should just use your own!
One thing that you'll find in that function that you should think about as well - we don't actually cluster every single frame, only frames with high motion energy. The idea is that the animal might spend a lot of time sitting still, and therefore there will be many redundant frames. So we filter out frames where there is not a lot of movement and only cluster frames where the animal is ostensibly moving, which gives us more diverse frames and is faster.

Hi, @themattinthehatt
here is the code I tried.
%%time
#GPU
import cv2
import numpy as np
import torch
import logging
from torch import nn
from sklearn.base import BaseEstimator, ClusterMixin
from tqdm import tqdm

logging.basicConfig(level=logging.INFO)

# CPU-based frame resizing
def resize_frame_cpu(frame, size=(256, 256)):
    resized_frame = cv2.resize(frame, size)
    return resized_frame

# CPU-based frame conversion to grayscale
def convert_to_grayscale_cpu(frame):
    gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    return gray_frame

# Read video frames
def read_video(video_path):
    cap = cv2.VideoCapture(video_path)
    frames = []
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        frames.append(frame)
    cap.release()
    return frames

# Process video frames using CPU
def process_frames_cpu(frames, batch_size=5):
    processed_frames = []
    for i in tqdm(range(0, len(frames), batch_size), desc="Processing frames"):
        batch_frames = frames[i:i+batch_size]
        for frame in batch_frames:
            try:
                resized_frame = resize_frame_cpu(frame)
                gray_frame = convert_to_grayscale_cpu(resized_frame)
                processed_frames.append(gray_frame.flatten())
            except Exception as e:
                logging.error(f"Error processing frame: {e}")
                continue
    return np.array(processed_frames)

# Custom K-means implementation using PyTorch
class KMeansModel(nn.Module):
    def __init__(self, n_clusters, max_iter=100):
        super(KMeansModel, self).__init__()
        self.n_clusters = n_clusters
        self.max_iter = max_iter

    def forward(self, X):
        device = X.device
        indices = torch.randperm(X.size(0), device=device)[:self.n_clusters]
        centroids = X[indices]

        for _ in range(self.max_iter):
            distances = torch.cdist(X, centroids)
            labels = torch.argmin(distances, dim=1)
            new_centroids = torch.stack([X[labels == j].mean(dim=0) for j in range(self.n_clusters)])
            if torch.all(centroids == new_centroids):
                break
            centroids = new_centroids

        return labels, centroids

# Skorch wrapper for the custom K-means model
class SkorchKMeans(BaseEstimator, ClusterMixin):
    def __init__(self, n_clusters, max_iter=100, device='cpu'):
        self.n_clusters = n_clusters
        self.max_iter = max_iter
        self.device = device
        self.model = KMeansModel(n_clusters, max_iter).to(device)

    def fit(self, X, y=None):
        self.model.train()
        X = X.to(self.device)
        self.labels_, self.centroids_ = self.model(X)
        return self

    def predict(self, X):
        self.model.eval()
        X = X.to(self.device)
        with torch.no_grad():
            labels, _ = self.model(X)
        return labels.cpu().numpy()

# Main function to integrate everything
def main(video_path, n_clusters=5):
    logging.info("Reading video frames...")
    frames = read_video(video_path)
    logging.info(f"Read {len(frames)} frames")

    logging.info("Processing frames on CPU...")
    processed_frames = process_frames_cpu(frames, batch_size=5)
    logging.info(f"Processed {processed_frames.shape[0]} frames")

    # Convert processed frames to a format suitable for PyTorch
    processed_frames_torch = torch.tensor(processed_frames, dtype=torch.float32)

    # Check if CUDA is available and set the device
    device = 'cuda' if torch.cuda.is_available() else 'cpu'
    logging.info(f"Using device: {device}")

    logging.info("Performing K-means clustering...")
    kmeans = SkorchKMeans(n_clusters=n_clusters, max_iter=100, device=device)
    kmeans.fit(processed_frames_torch)
    labels = kmeans.predict(processed_frames_torch)
    logging.info(f"Clustering completed. Cluster labels: {labels}")

if __name__ == "__main__":
    video_path = "videos/xxxxx.mp4"
    main(video_path)

I just asked chatgpt to generate the code and found that it can run on GPU. I haven't  check in detail yet. please be careful about it. By the way, there is another way recommended is the cuML in RAPIDS, I have tried cuML a few years ago on single cell sequencing data(high dimensional data), it worked perfectly. But haven't try cuML on the behavior data yet.

thanks for sharing, I'll definitely look into this!

Hi, @themattinthehatt
just to update, I highly recommend cuml for now.

I check the code in detail and found that it is easy to overwhelm the GPU memory with a lot of GPU accelerated methods (that means it is good to run on toy data, but might crash if run on real video data).
rapids method like cuml has special advantage on GPU memory management by rmm(RAPIDS Memory Manager).
to handle one of my real video datasets, with same hardware and same setting(like 100 clusters, max_iteration=100...), scikit-learn takes 8min48s, and cuml takes 28.6s.
4.my cuml code:

%%time
import cupy as cp
import cuml
from cuml.preprocessing import StandardScaler
from cuml.cluster import KMeans

# Assuming features is already defined and available as a NumPy array

# Scale features
scaler = StandardScaler()
features_scaled = scaler.fit_transform(features)

# Transfer scaled features to GPU
features_scaled_gpu = cp.array(features_scaled)

# KMeans with cuML
kmeans = KMeans(n_clusters=100, init='k-means++', max_iter=100)
kmeans.fit(features_scaled_gpu)

# Get the cluster centers and labels
centers = kmeans.cluster_centers_
labels = kmeans.labels_

print("Cluster centers:", centers)
print("Assignments:", labels)

cool, that's quite the speedup! do you seem to get a diverse set of frames from this procedure?

cool, that's quite the speedup! do you seem to get a diverse set of frames from this procedure?

Hi, @themattinthehatt
what do you mean by 'a diverse set of frames'?

You're doing this frame extraction for labeling new frames, right? I'm curious if you extract 100 frames in this way if they all look visibly different from each other - the mouse is in different parts of the arena, in different poses, etc.

You're doing this frame extraction for labeling new frames, right? I'm curious if you extract 100 frames in this way if they all look visibly different from each other - the mouse is in different parts of the arena, in different poses, etc.

Hi, @themattinthehatt I checked the video and found that poses, position, location seem to all contribute to the clustering, and I could not tell which one would contribute more if only by eyes.
and I check your code and think it is more reasonable since you include pca and focus on high dynamic state.
can you give more details about how to make use of those functions in extract_frames.py?

Hi, @themattinthehatt
I check extract_frames.py in details and found it is very good. I would suggest to include this functions not only in LP app but also in LP itself. But still if you can offer more details or examples in the LP tutorial, that would be pretty good. I just want to make sure I do not have misunderstanding about your code.
and by the way in the find_contextual_frames functions, if the center frame is 1, then it would get [0,1,2,3] (-1 is excluded), and this [0,1,2,3] is not five-frame chunk, so will it still be included in the LP context model?

The find_contextual_frames function checks for groups of consecutive frames. If a group has 5 or more consecutive frames, it removes the first two and last two frames from that group.
what purpose is this function for?

@Wulin-Tan glad you found the functions in extract_frames.py helpful. We've considered putting them in the LP repo itself but since we don't also offer labeling tools in LP we decided to group all of the non-training/inference code together in the app.
Re find_contextual_frames - our documentation is not so good yet with the app code. This function is actually for when people are uploading zipped folders of frames into the app for labeling - we want to check if they have included context frames or not. If we decide context frames have been included, this function returns only the frames that need to be labeled, and not their context, so that we can upload the proper frames into label studio. For the specific case you raised when the center frame is 1, the LP code will see that the first context frame should be -1 and just loads a second copy of frame 0 instead, so there will be no errors.
One question for you - are you attempting to label 500 frames from a single video? If so one piece of advice I have is to label fewer frames from more videos than many frames from a single video - this leads to better results on new animals. So if you're labeling budget is 1000 frames, it is better to label 50 frames from 20 animals rather than 500 frames from 2 animals.


Push arm64 docker images to DockerHub

We produce arm64 packages for Linux. We should be pushing an ARM64 docker image as well. We already have dokken images ready for them.

This is waiting on this functionality within our pipelines. https://github.com/chef/release-engineering/issues/1344

Done


github.com/deprecated-rs maybe?

Why github.com/deprecated? Should this namespace cover other languages than Rust?
If Rust only, then it's better to name it github.com/deprecated-rs.

It's subtle, but the org name is deprecrated, i.e. it's a list of deprecated crates. Seems appropriate to me 😃

It's subtle, but the org name is deprecrated, i.e. it's a list of deprecated crates.

Exactly


Issue with percentage bar

The percentage bar is not working correctly after calling setPercentage().

Fixed in version 0.2.2.


Decoding of records with missing fields.

This is a test implementation of decoding records where some fields may not exist in the encoded data.
Before giving up decoding it will attempt to run the decoder for any missing field with a 0 byte input: Decode.decodeWith [] Decode.decoder fmt and if the decoder returns a success it will put that in the field.
This allows us to define types that have specific behavior when decoding nothing. eg:
Option val := [None, Some val]
#...
optionDecode = Decode.custom \bytes, fmt ->
    if bytes |> List.len == 0 then
        { result: Ok (@Option (None)), rest: [] }
    else
        when bytes |> Decode.decodeWith (Decode.decoder) fmt is
            { result: Ok res, rest } -> { result: Ok (@Option (Some res)), rest }
            { result: Err a, rest } -> { result: Err a, rest }

As we can see, if no bytes are given we just output a "None" value

A full code example:
Option val := [None, Some val]
    implements [
        Eq {
            isEq: optionEq,
        },
        Decoding {
            decoder: optionDecode,
        },
    ]

none = \{} -> @Option None
some = \a -> @Option (Some a)
isNone = \@Option opt ->
    when opt is
        None -> Bool.true
        _ -> Bool.false

optionEq = \@Option a, @Option b ->
    when (a, b) is
        (Some a1, Some b1) -> a1 == b1
        (None, None) -> Bool.true
        _ -> Bool.false

optionDecode = Decode.custom \bytes, fmt ->
    if bytes |> List.len == 0 then
        { result: Ok (@Option (None)), rest: [] }
    else
        when bytes |> Decode.decodeWith (Decode.decoder) fmt is
            { result: Ok res, rest } -> { result: Ok (@Option (Some res)), rest }
            { result: Err a, rest } -> { result: Err a, rest }

# Now I can try to modify the json decoding to try decoding every type with a zero byte buffer and see if that will decode my field
OptionTest : { y : U8, maybe : Option U8 }
expect
    decoded : Result OptionTest _
    decoded = "{\"y\":1}" |> Str.toUtf8 |> Decode.fromBytes TotallyNotJson.json
    dbg "hil"

    expected = Ok ({ y: 1u8, maybe: none {} })
    isGood =
        when (decoded, expected) is
            (Ok a, Ok b) ->
                a == b

            _ -> Bool.false
    isGood == Bool.true
OptionTest2 : { maybe : Option U8 }
expect
    decoded : Result OptionTest2 _
    decoded =
        """
        {"maybe":1}
        """
        |> Str.toUtf8
        |> Decode.fromBytes TotallyNotJson.json
    dbg "hil"

    expected = Ok ({ maybe: some 1u8 })
    expected == decoded

Testing this PR with the below example and a minor update to roc-json to pass arguments to the finaliser.
# switch to PR branch
cargo run -- run optional-decoding.roc
(Ok {count: 12, optional64: (@Option None)})

app "optional-decode"
    packages {
        cli: "../basic-cli/platform/main.roc",
        json: "../roc-json/package/main.roc",
    }
    imports [cli.Stdout, json.Core.{ json }]
    provides [main] to cli

main =

    input = "{ \"count\": 12 }" |> Str.toUtf8

    decoded : DecodeResult { optional64: Option U64, count: U32  }
    decoded = Decode.fromBytesPartial input json

    decoded.result
    |> Inspect.toStr
    |> Stdout.line

Option val := [None, Some val] implements [
    Decoding { decoder: optionDecode },
    Inspect, # auto derive
]

optionDecode : Decoder (Option val) fmt where val implements Decoding, fmt implements DecoderFormatting
optionDecode = Decode.custom \bytes, fmt ->
    if bytes |> List.isEmpty then
        { result: Ok (@Option (None)), rest: [] }
    else
        when bytes |> Decode.decodeWith (Decode.decoder) fmt is
            { result: Ok res, rest } -> { result: Ok (@Option (Some res)), rest }
            { result: Err a, rest } -> { result: Err a, rest }

Tests are currently failing because the basic-cli has an "Env" decoder in it and needs to have the type annotation for it updated to reflect the fact that finalize has 2 params now

Are you ok with this change in general @rtfeldman, see also zulip? If so, I'll do the full review of this, do a new basic-cli release, etc. .

I'm ok with it if @ayazhafiz is ok with it! 👍

This approach overall looks good. I would really prefer to have the implementation all in one file. It would make the diff easier to review and I don't think there is any benefit in splitting up the derivation across multiple files.

I find enormous singe files make it annoying to navigate through and find the parts I want, and I like how breaking things up makes it more clear when things are isolated from each other or dependent.
But yeah it does make reviewing harder. I did do the splitting at the very end so I'm very happy to recombine if you like, or I can do the splitting in another PR. Just let me know :)

What do you think of @faldor20's suggestions @ayazhafiz?

@faldor20 let’s keep everything in one file for now and address the split in another PR. I think it’s helpful to keep the size of the PR small, especially for future contributors given that deriving is pretty verbose already.

okay @ayazhafiz hopefully that's all good. It will still be failing CI because we need to merge this: https://github.com/roc-lang/basic-cli/pull/176

I'm going to merge this Monday morning, because the tutorial is already updated for basic-cli 0.9.0, but that won't work unless you have Monday's nightly.

@Anton-4 Thanks a lot for all your work getting this deployed everywhere :)


KeyError: 'SNORKELHOME'

The following error occurs when vectorizing the .tsv data file (i.e. corpus_parser.apply(doc_preprocessor)).
/Users/snorkel/snorkel/parser.pyc in __init__(self, tok_whitespace, split_newline, parse_tree)
    196         self.split_newline = split_newline
    197         self.parse_tree = parse_tree
--> 198         loc = os.path.join(os.environ['SNORKELHOME'], 'parser')
    199         cmd = ['java -Xmx4g -cp "%s/*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer --port %d --timeout %d > /dev/null'
    200                % (loc, self.port, 600000)]

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/UserDict.pyc in __getitem__(self, key)
     38         if hasattr(self.__class__, "__missing__"):
     39             return self.__class__.__missing__(self, key)
---> 40         raise KeyError(key)
     41     def __setitem__(self, key, item): self.data[key] = item
     42     def __delitem__(self, key): del self.data[key]

KeyError: 'SNORKELHOME'

Hey @Tsmith5151, looks like your SNORKELHOME environment variable isn't set. Running either the run.sh script or source set_env.sh in the top-level snorkel directory should set it for you.

Hi @henryre thanks for pointing this out. On another note, when loading in the labels from gold_labels.tsv for the testing set in the intro tutorial, the number of "AnnotatorLabels" created is 0. Looks like it should be 220/223 and 273/279 labels loaded?
from load_external_annotations import load_external_labels
load_external_labels(session, Spouse, annotator_name='gold')

Hi @Tsmith5151, if you run load_external_labels(session, Spouse, annotator_name='gold') and then rerun it a second time, the labels will not reload into the database (which is a good thing!), so that's the expected behavior. You can verify that there are 220 labels in the dev set by running
from snorkel.annotations import load_gold_labels
L_gold_dev = load_gold_labels(session, annotator_name='gold', split=1)
print "The dev set has {0} labels".format(np.sum(L_gold_dev.todense() != 0))

This is a good point to clarify though, so thanks for pointing it out. Closing this issue for now. If you run into any other issues, feel free to post a new issue.

Hi @henryre, is snorkel currently compatible to process searchable PDF documents?


Incompatibility with Lissi OOB URLs

I've recently trying out Lissi demo to see how far an AFJ-based app can go and I noticed that in their invitation URLs they are using oob query parameter and encoding a connections/1.0/invitation message into it. AFJ expects it to be an out-of-band/1.x/invitation message, so as a result it throws a validation error when scanning the code.
Whilst I think AFJ interpretation is correct, based on the wording from Aries RFC 0434 where it says The oob query parameter is required and is reserved to contain the out-of-band message string one could also interpret that 'out-of-band' refers to any DIDComm message that will be send out of band (i.e not strictly from the out-of-band protocol).
So in some way I don't see that it would be harmful to allow processing a former connections protocol invitation encoded in oob query parameter. What do you think?

Yeah makes sense! No harm in supporting it I think


Greenkeeper/@types/node 10.3.1

Coverage remained the same at 40.876% when pulling 1f3f2f89bf3db791b94aeb45be30bc1b01451b66 on<EMAIL_ADDRESS>into fafdd931cd0ea90f53c7b9adc71972b9abfc2327 on master.


Add gscan to gulp deploy as validator and use package (theme) name within the deploy task

Using gscan, the gulp task deploy will now generate a zip file and then scan the zip file for compliance with ghost. Any issues will be printed to the console. This implimentation borrows heavily from gscan's cli implimentation, which can be found at gscan/bin/cli.js.
When the deploy task is run, Gulp will pull in the name property from the package.json file and use it as the name for the generated zip file.
The scan task has also been updated to use this file name.

Here is an example of the scan's output.


Improve error reporting

Force file location, even when using pythran in library mode
Improve file location for spec file

Related to https://github.com/scipy/scipy/issues/13308
Fixes #1690

Related to https://github.com/scipy/scipy/issues/13308
Fixes #1690


com.google.fonts/check/whitespace_ink: NotoSansOgham fails

Ogham fonts require its space character to have ink in most cases. This is a bug.

U+1680,  , OGHAM SPACE MARK, is the only Unicode whitespace character that typically has ink.
This being the case however, I think this check should be downgraded to a warning. Some fonts will want ink in normal U+0020 whitespace, e.g. FRBAmericanCursive, in its fonts which have guidelines.

This was a bug in a python expression lacking parentheses. This is fixed now. Thanks for reporting ;-)

I will keep the check at FAIL level for now. I looked up FRBAmericanCursive, and its usage of a guideline is clearly a corner-case exception.


New "tiles" SQLite table structure

Issue Type

[x] Improvement
[x] Performance

Description and/or steps/code to reproduce the problem
Currently, the tiles table in SQLite tile cache database has a 2-column primary key.
One column (provider) is the tile source (e.g. 'Mapnik').
The other column (key) is a long where the zoom, x and y values are packed.
We may be better off with the following organization:

one table per zoom level (30 tables, from 0 to 29)
x and y as part of the primary key (along with provider)

Pros:

one table per zoom level means smaller tables, therefore better performances for queries and admin tasks
x and y columns make it possible to run queries about a bounding box with SQL syntax (where provider='Mapnik' and x between 4 and 8 and y between 6 and 10), which we cannot perform now. A typical use case is: "I have to purge the cache but I want to keep some bounding boxes / I specifically want to remove some bounding boxes"
a table per zoom level makes it possible to delete data in truncate mode (This "truncate" optimization makes the delete run much faster). Use case: "I have to purge the cache, let's remove the tiles on zoom levels > 17"
self explanatory columns (x, y, provider, tile, expires in table tiles_zoom_12)

Cons:

it will take a little bit more admin space (30 tables instead of 1; bigger primary key; possibly additional indices on x and on y for better performances)
we have a whole history to migrate

Impact:

the impact on the queries would be limited to SqlTileWriter

Version of osmdroid the issue relates to:
6.0.0

@monsieurtanuki I know you have a run right now, and from distance it looks like many things get better, but no one gets hold of it, as there is no release. I.e. I'd really like to have that faster cache update in my app! So, just a few thoughts:

consider this as change not before 6.2.0?!
I do not see the relevancy of truncate in a real world cache scenario.
I see some practical applications for "delete bounding box", I have no idea how without lots of handcrafting in the surrounding app we will have a "keep bounding box". So far it is simple and working good enough, to be just ignored in many use cases.
Do we really expect much gain from 30 tables compared to one table with zoom as a column?
Then why not 30 databases, shouldn't that give parallel access? Like deleting cache preferably in the zoom not currently displayed or something alike?
Migration will be a real problem to explain to the user. For example I am very focused to have my app start up quickly.
Maybe just provide it as an alternative sql cache, so that the app developer can decide which one to plug in and when to migrate (well makes more code to be maintained)?

@InI4

I don't plan to deliver this for 6.1.0 (cf. #1001)
"truncate in a real world use case" - my Android smartphone keeps telling me that I don't have enough space in order to upgrade all my apps, therefore I need space, and with the suggested architecture I can easily know how much data there is in each zoom level (which is not implemented now) and as the highest zoom levels contain more data and are possibly less useful (depends on the actual use case), I can decide to purge quickly, relevantly and with the maximum space removed
"keep bounding box": I need space, and I know I can easily get rid of all data but I want to keep Paris, therefore I run something like delete from tiles_12 where not (x between 567 and 896 and y between 345 and 456)
"expect much gain from 30 tables" - I have no figures, but each database operation should be faster (insert, update, delete, select) because the tables will be smaller and so will the related admin operations (e.g. the recent index creation). You're right, I need to run performance tests.
"why not 30 databases" - databases are designed to host many tables, and Android is not designed to run many databases at the same time
"Migration will be a real problem" - definitely. But will there be one? (see below)
"provide it as an alternative sql cache" - could be the solution, like when osmdroid migrated from files to database. @spyhunter99 Was there actually a migration, or just a "from now on we use the database instead"?

Today we have something like that in MapTileProviderBasic:
IFilesystemCache tileWriter = (Build.VERSION.SDK_INT < 10) ? new TileWriter() : new SqlTileWriter();

We could imagine a test inside SqlTileWriter's constructor: if table tiles exists we run things as usual, if it doesn't exist we are in a fresh new database and can create the 30 tables instead. And maybe not (never?) migrate.

I just run some performance tests on my smartphone with both tiles table version (the current one, and a zoom-focused one with columns x and y instead of key, one index on x and one index on y - a.k.a. the "issue" version - there will 30 tables like this one):

create the table for the very first time
Takes about 23ms in the current version, 36ms in the issue version (to be multiplied by 30, therefore 1080ms)
=> I have 2 additional indices that would explain the speed difference. Maybe I can create the indices in the same query as the table creation, in order to save time (to be checked)
try to create the table when it already exists
Takes about 1ms in the current version, 1ms in the issue version (to be multiplied by 30, therefore 30ms)
=> It would be therefore harmless to use the issue version
insert N tiles (N being 625, 2500 and 10000) (with a 10Kb blob tile)
Regardless of N, it takes about 12.03ms by tile in the current version, 12.52ms in the issue version
=> slightly slower because of additional indices
replace N tiles (same N)
Regardless of N, it takes about 17.40ms by tile in the current version, 18.33ms in the issue version
=> slightly slower because of additional indices
create the index on expires
In both versions, the bigger the table (N), the slower the index creation, not only as a whole operation, but also as when divided by N. But the index creation is maybe too fast for performance relevancy (about 16ms for N=625, about 400ms for N=10000)
=> the bigger the table, the slower the index creation
run 50 times a select of 1 item that exists
In both versions, the bigger the table the slower the select, but we're always dealing with about 1ms operation.
=> fast select in both versions
run 50 times a select of 1 item that does not exists
In both versions, the bigger the table the slower the select, but we're always dealing with about 1ms operation.
=> fast select in both versions
run 50 times a delete of 1 item that exists
Regardless of the table size (N) and of the version, it takes about 20ms by tile
=> same item deletion time in both versions
run 50 times a delete of 1 item that does not exists
Regardless of the table size (N) and of the version, it takes about half a ms by tile
=> same item deletion time in both versions
bulk delete of half the table
Regardless of the table size (N) and of the version, it takes about 1.5ms by tile
=> same deletion time in both versions
delete/truncate the table
I have very unexpected results.
In the current version, the truncate gets slower (!) per N (from 0.25 for 625 to 0.97 for 10000)
In the issue version, the truncate gets faster per N (from 0.25 for 625 to 0.17 for 10000)
=> I expected similar results for both versions. I think I need to run again the N=10000 performance tests.

I run again some tests, and I still have a truncate operation 5 times slower with the current version! Very surprising... But that's not the main focus of this issue.
I had to check if we could run several SQL statements at the same time. We cannot by putting several semicolon in the same execSQL statement, but (and that's a good surprise) we can put DDL statements inside a transaction, which makes things much faster. Creating 30 tables with 2 indices took about 200ms.
Performance comparison tests conclusions:

create the table for the very first time
=> takes about 20ms in the current version, 200ms in the issue version. Acceptable for a one-shot operation.
try to create the table when it already exists
=> same time
insert a tile
=> slightly faster in the current version because of additional indices
replace a tile
=> slightly faster in the current version because of additional indices
create the index on expires
=> the bigger the table, the slower the index creation
select of 1 existing item
=> same time
select of 1 non-existing item
=> same time
delete 1 existing item
=> same time
delete 1 non-existing item
=> same time
bulk delete of half the table
=> same time
delete/truncate the table
=> for some reason it's much slower in the current version

Therefore:

there's no obvious performance interest in changing table structures, except for late admin tasks like create index
I focus now on the functional interest, and try to write SQL statements with the current table structure that would emulate the "zoom=17" or bounding box selection

Just found an effective way to perform "zoom=17" or bounding box selections.
Now I need to make it easy to use.
Should PR by the end of the week.


Mysql backup raises warning

Hi,
I am doing a backup of a MySQL database and since I am providing a password (there is no other way to connect to a Mysql database), it raises a warning:
[2016/03/16 04:00:02][info] Database::MySQL Started...
[2016/03/16 04:00:02][info] Using Compressor::Gzip for compression.
[2016/03/16 04:00:02][info]   Command: '/bin/gzip --rsyncable'
[2016/03/16 04:00:02][info]   Ext: '.gz'
[2016/03/16 04:00:02][warn]   Pipeline STDERR Messages:
[2016/03/16 04:00:02][warn]   (Note: may be interleaved if multiple commands returned error messages)
[2016/03/16 04:00:02][warn]
[2016/03/16 04:00:02][warn]   Warning: Using a password on the command line interface can be insecure.
[2016/03/16 04:00:02][info] Database::MySQL Finished!

hi @andrefreitas, thanks for reporting. Considering you already closed the issue can I assume you found the options/password file mysql recommends you put your passwords in?

@tombruijn  @andrefreitas - same issue, please reopen. Anybody found solution?

@Bogdaan yes, please use a .my.cnf file like so:
[mysqldump]
user=root
password="pass"

Don't set your username/password in the backup config but in that file instead.
Read more about that mysql config file here: http://dev.mysql.com/doc/refman/5.7/en/option-files.html

@tombruijn
thanks for reply, but i dont use ROOT !
I have multiple mysql databases & access - and always get this "warnings"

I don't understand what using root or not has to do with this? It's just an example, replace that username and password with your own database's config.
You create a .my.cnf file and pass it in to mysqldump through the additional_options in the model config.
I need more information before I get what you're trying to do and what your current issue is


Issue2023curlybracket

Pull Request for #2023
Issue2023curlybracket
This code currently raises a warning and detects the following Extra Brackets:

OPEN_ROUND_BRACKETS
CLOSE_ROUND_BRACKETS
OPEN_CURLY_BRACKETS
CLOSE_CURLY_BRACKETS

and outputs the warning number 1016.
In order to remove the warning, the user would need to look at the warning message showing the line number of the extra bracket.
Note

For Developers, Quick Build the Compiler twice as there is an edit in the Umple Internal Parser Code.
Multiple Test Cases has been added to test and solve this issue.
An extra user manual guideline has been added to show how to reproduce this warning and how to solve the warning.

Reasons for commits:
Test Cases accidentally all have Class instead of class.
An example would be:

Class x{}
It was changed to:
class x{}

Jenkins Giving Failed Warnings.

It does not pass Jenkins as Jenkins only build the compiler once
Since there is a change in the internal parser code, the compiler should be compiled twice in order for brackets to be detected.


Coq commands don't work in Goal and Info buffers

Commands like :Coq Check and :Coq Locate etc. (and their corresponding keybinds) are local only to the buffer where the Coq script is, but it's often useful to have these commands to explore the proof context and output buffers.
Right now, the workaround I have is to yank whatever term I need into a register, switch my window focus back over to the Coq script, and either run :Coq Check <c-r>" or paste it into the Coq script and manually invoke the vernacular.

I agree that this would be nice to have. What's blocking it is that currently a lot of the code has the assumption that it's called from the main buffer baked in. That was mostly just a simplifying assumption to get things working initially but it shouldn't be too hard to remove now.


Garbage Collection not visible in Integrations

Before you submit a new bug report, please check that

[X] I have read frequently asked questions and limitations

Write your question here
Hi Vaclav, thanks for the cool integration, I'm using it with the custom-button card in lovelace and looks good and working fine.
One challenge is that though it's installed and working properly I can't see the Integration shown under Integrations.  I added new garbage collection sensors & schedule etc through both Settings > Devices > Helpers page and through Settings > Devices > Integrations page.  But I assume I should be able to see the Integration under the Settings > Devices > Integrations, and hence I can't seem to delete any collections or edit schedules for existing ones?
Any suggestions?  No stress if not, since it is working fine.  Thanks

That's correct, the helpers do not show  under Integrations, but under the helpers tab. You can manage them from there.

That's correct, the helpers do not show  under Integrations, but under the helpers tab. You can manage them from there.

Ok understood and managed to delete old ones from helpers.  Tks

Thanks


add validation to check against newly created old-style multipolygons

following up on #3908 and https://github.com/osmlab/fixing-polygons-in-osm/issues/27#issuecomment-289816177, it would make sense to add a validation rule that checks whether newly created multipolygons follow the modern tagging scheme (i.e. tags on the relation, not outer ring) – e.g. similarly how we currently warn if a user tries to upload an untagged closed way (with area=yes).

AFAIK, iD only creates the new kind of multipolygon when merging line-line or line-area.
For iD to create an old style multipolygon, I think someone would need to manually add a tagged outer ring  to a new multipolygon relation.  In that case, rather than validate, maybe we should just silently move the tags from the outer to the multipolygon when the actionAddMember or actionChangeMember step happens?    I think we can do this with actionChangePreset, so that only the important tags get changed.
I kind of doubt that this check makes sense as a validation because:

it's not likely that people are manually creating multipolygons
it's somewhat hard to fix - I fixed a few old legacy multipolygons in iD yesterday, but I don't think most people would know to select the multipolygon, assign the preset, select the outer, delete the relevant tags.

It's worth mentioning that we also don't show users the right way to do it.   The walkthrough has been on my mind this week, and I'm thinking of maybe adding a second "Advanced Editing" walkthrough that could teach how to create multipolygons with the merge operation among other things.

For iD to create an old style multipolygon, I think someone would need to manually add a tagged outer ring to a new multipolygon relation.

That's also what I think is the case.

it's not likely that people are manually creating multipolygons

Well, a few people apparently do (see https://github.com/osmlab/fixing-polygons-in-osm/issues/27#issue-217015183). Maybe intentionally, maybe following out-of-date instructions?

just silently move the tags from the outer to the multipolygon when the actionAddMember or actionChangeMember step happens?

Hmm. That could be convenient. Still, I think that there could/should be a validation error in case the created multipolygon still doesn't have tags (e.g. when created from a fresh area=yes polygon or an open ring segment): I think it'd be quite logical that if we warn when one saves one of these area=yes ways, to also warn in the equivalent multipolygon case.

In that case, rather than validate, maybe we should just silently move the tags from the outer to the multipolygon when the actionAddMember or actionChangeMember step happens?

I think we can't do that. Multipolygon members can have their own properties. We don't know which tags the user intends to add later. An automatic decision to transfer tags based on incomplete data is not viable.
Valid use of area properties on outer members is rare, except of old style multipolygons. Therefore, iD might ask the user if tags should be transfered in case of adding an way with area tags as outer to an untagged multipolygon.
We can clean up automatically before saving to OSM, because at this stage the user should have added all intended information:
In case the multipolygon is untagged, and all outers do have the same tags, and the tags are not valid for lines, we can move the tags to the multipolygon.
We should also do another cleanup step here:
In case a multipolygon has no inner members, convert it into a closed way area, provided that the outer members meet the conditions:

are forming a single ring
aren't tagged
aren't member of any other relation
don't have too much nodes in total

Well, a few people apparently do (see osmlab/fixing-polygons-in-osm#27 (comment)).

Are these really valid old style multipolygons? Do their members have tags which are suitable for areas only? Do they have more than one member?
Maybe these are accidentially created multipolygons with a single member, or untagged features with line properties on its members.

Let's not do anything for this.  We can reopen if there is data that shows a lot of users are creating old-style multipolygons with iD, but for now I don't think it's a common enough thing to spend time on.

The existing old-style multipolygons have been fixed. Pretty much all the newly creating old-style multipolygons that get added on a daily basis are by iD users.
Please reopen this issue and fix iD to do the right thing. It is a terrible decision not to handle multipolygons properly for the default editor on osm.org.

The existing old-style multipolygons have been fixed. Pretty much all the newly creating old-style multipolygons that get added on a daily basis are by iD users.

Sure, if this is true I would be happy to add a validation, otherwise I would prefer to spend my time working on improvements to iD that benefit more users.  Do you have examples?

Some examples:

http://www.openstreetmap.org/changeset/48355535 (e.g. relation 7212724)
http://www.openstreetmap.org/changeset/48325271 (e.g. relation 7210310)

Thanks @sebastic, it's helpful to know that people are actually doing this daily.  I'll add the validation 👍

And thank you for reconsidering this issue.

Ok I added this.  It looks like this now, with a warning and a tooltip explaining how to fix the issue:

Looks good. The validation may be too simple for cases where some tags on the outer way are appropriate like barrier=fence and the relation should have landuse=*.

Looks good. The validation may be too simple for cases where some tags on the outer way are appropriate like barrier=fence and the relation should have landuse=*.

This warning will appear if the relation has no interesting tags and the outer way has some interesting tags.
We can't give guidance to the user about which tags belong where, but I think it's fair to assume that anyone who creates multipolygons this way should be able to figure it out.
A validation like "fences belong on ways and landuses belong on multipolygons" is really out of scope for an editor, but would be a great thing to add to any of the dedicated OSM Q/A tools.

I think you're overestimating iD users, experienced users who know what they're doing tend to use JOSM.
This warning is already bettter than nothing. And we can't expect iD to every become as full featured as JOSM. So I guess we should leave it at this.

@sebastic iD generates itself no old-style polygons. So what bhousel says: When someone was able to build an old-style polygon with iD it was on purpose and he should be able to fix it.

iD users are generally not experienced mappers, and are not able to fix these issues without assistance.
It's the Potlatch story all over again. Inexperienced mappers create issues because the default OSM editor does not help them do the right thing.

iD does help them do the right thing - multipolygons it automatically creates when forming a geometry that requires them are new-style multipolygons. Just like with any editor which allows manually editing relations, it's possible to create an old-style MP.
If they're still creating old-style MPs, we need to know why to know the appropriate changes.

But iD doesn't help them by providing an advanced validator like JOSM which does a much better job at helping users do the right thing.
I'm leaving this conversation now, as iD will never live up to my expectations of a decent editor. Thanks for 660511e, that is a very welcome change.

@pnorman, I was called out by @sebastic for creating old-style multipolygons. I did it, among other reasons— because I didn't know any better, because iD never warned me about it, and because the multipolygons I had created appeared to work just fine.
Reading the comments here, the perception seems like a user must go out-of-their-way to create old-style multipolygons. Maybe. But for me, iD seems to make it easy. Here was my old process to create a large pool deck surrounding a swimming pool:

Draw two areas: a smaller area nested in a larger area.
Tag the areas, respectively, as swimming pool and pedestrian street.
In the Edit Feature Panel for the pedestrian street, add a new multipolygon relation; tag the pedestrian street as "outer."
Add the newly created multipolygon relation to the swimming pool; tag the swimming pool as "inner."

A few points:

The problem, for me at least, was the option to create a new multipolygon relation on an area. That a multipolygon requires a "line" as outer is not intuitive. When I think "multipolygon," I think "multiple areas." I think, "small area nested inside big area." Or some kind of "mainland/ exclave" relationship.
The process for creating new-style multipolygons is not made clear in iD.
It appears that I had "manually" created these old-style multipolygon relations. Was there a more "automatic" process in iD that I missed?

@fluous The more automatic way to create multipolygons in iD is to draw the two areas, (or an outer area and inner line for a hole) - then select them both and use the "merge" command.
We know this isn't intuitive yet, see #3134.  We also plan to train users on this technique eventually by offering a more advanced walkthrough, see #4012


Revert commit that rolled back logger.Dockerfile, never push :latest

What this PR does:
Which issue(s) this PR fixes:
Fixes #
Checklist

[ ] Changes manually tested
[ ] Automated Tests added/updated
[ ] Documentation added/updated
[ ] CHANGELOG.md updated (not required for documentation PRs)
[x] CLA Signed:  DataStax CLA

For context, this is where the version-tag is used for system-logger and cass-operator to ensure our tests don't just use :latest and be happy:
https://github.com/k8ssandra/cass-operator/blob/master/Makefile#L208

Ah.. need to fix the Kustomize version again:
/home/runner/work/cass-operator/cass-operator/bin/kustomize version is not expected v5.0.3. Removing it before installing.
test -s /home/runner/work/cass-operator/cass-operator/bin/kustomize || { curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash -s -- 5.0.3 /home/runner/work/cass-operator/cass-operator/bin; }
Github rate-limiter failed the request. Either authenticate or wait a couple of minutes.


Unable to delete or push tag

Sublime Text version number: Build 3159 (happens under previous versions also)
Git version number: 2.14.2
OS type and version: MacOS 10.13.3
Console error output:

`/usr/local/bin/git tag -d 7 v1.77` failed with following output:

error: tag '7 v1.77' not found.

A description of the problem:
I found the [d] delete and [p] push to remote commands cannot properly parse the tag. For tags displayed as below in TAGS view:

  LOCAL:
    1160e4b17 v1.77
    4a719e149 v1.73
    ... ...

delete and push commands will always grab the last character of the commit id + the tag id (e.g. 7 v1.77,  9 v1.73), but [l] view commit command can correct parse and display the tag

It seems that the commands in the tag dashboard have assumed 8-digit hashes. A workaround for you at the moment is to force core.abbrev to be 8
git config --local core.abbrev 8


Fix collecting of still used weak references

If the RecyclerWeakReference has been marked, it must not be removed just because the object it refers to will be collected.

This map is used only for setting the strongRefs to nullptr, so the weak ref will not be collected, because it is not present.


Chrome Store release

When the addon gets 'stable' it would be wonderful to release this to the mass within the Chrome Store.

Yes there is a plan to release it to the market. We haven't do so already, mainly because as you say the addon is not very stable yet.
This is a good idea... I will actually open a ticket and see where the discussion goes...


After helm upgrade to scale diego cell count from 3 to 4, 2 out of 3 diego-cell pods gets restarted

Describe the bug
When I use helm upgrade to scale up the diego cell count for 3 to 4. diego-cell-2 and diego-cell-1 get restarted consecutively, once new diego-cell-3 is up.
diego-cell-0 never gets restarted
It takes 19 min approx for all pods to get in running state
To Reproduce

HA deployment (3 diego cells)
Deploy an app with 3 instances
run helm upgrade to bring the count of deigo cell to 4

Expected behavior
A new diego cell should appear, no pods should restart.
What happens
some of the old diego-cell pods (2 out of 3) get restarted
Environment
kubecf-2.0-RC1                  2.0-RC1
cf-operator-4.5.6+0.gffc6f942   4.5.6+0.gffc6f942
Additional context
diego-cell-0                             7/7     Running     2          3h6m
diego-cell-1                             7/7     Running     2          6m57s
diego-cell-2                             7/7     Running     2          10m
diego-cell-3                             7/7     Running     2          24m

I reproduced it. It seems to be restarted because the readiness probe fails. I will investigate more.

I verified that simply editing the StatefulSet and changing the value of REPLICAS restarts the pods. So we just need to decide wether this environment variable is needed in the spec. Some docs on where it's used here: https://github.com/cloudfoundry-incubator/quarks-operator/blob/master/docs/Features/rendering_templates.md#render-templates (look for REPLICAS).

Obviously the value of REPLICAS changes every time and that must be the reason all pods get restarted.

Yes, that is all expected (except, why did the bootstrap pod not restart? Does it not get the REPLICAS variable? (I know in SCF we had to handle this specially for the nats-0 pod too).
The question is though: why do we need that variable? Afaict there is no reference to is from KubeCF.
It also feels inconsistent that we provide AZ_INDEX and REPLICAS. Why does the pod need to know the total number of replicas, but not the total number of AZs?
@viovanov can you provide some background information?


Ethernet library - wrong ordering of begin parameters

The 'begin versions without MAC address parameter have new ordering of parameters.
You can't do this. There are other Ethernet libraries now which have begin without the MAC address parameter, but they use the same ordering of parameters as the version with MAC address.
Yes a network engineer would prefer ip, subnet, gateway, dns, but Arduino introduced in 2008 an ordering based on optional parameters defaults, where it is more likely to need to set a specific DNS IP and then a specific gateway more likely then a subnet other than the default. Doing this in different ordering doesn't make sense.

I found out that the Ethernet library is a copy of the STM32 Arduino Ethernet library and the wrong ordering originates there

PR https://github.com/arduino/ArduinoCore-renesas/pull/222


[Bug Report]: Correct incomplete resource removal

Describe the bug
Since the migration to the new dependency approach, the removal seems to struggle with the LAWs deployed by the Diagnostic Tests. Often a Resource Group with just the LAW remains of the tests:

To reproduce
Run the tests
Code snippet
No response
Relevant log output
No response

Even with the new approach the issue is unfortunately not resolved an requires further investigation

Issue is solved


Static pages aren't shown on the tag page

Static pages can be tagged in that admin interface but they aren't shown on the tag list. Maybe this is normal behaviour but I couldn't find any documentation to confirm that.
http://DOMAIN/tag/xxxx doesn't show the static pages that are tagged with xxxx
As a real world example: https://thepluralists.org/tag/open-culture should show a number of static pages but it does not.

Hi @thepluralistsorg 👋 This is expected, static pages are removed from all post lists as they are intended to be separate pages on your site that are linked to within the navigation or directly in your theme.
The only way you can pull static pages out as part of a post list is when using the {{#get}} helper with the appropriate filters.
Closing as this isn't a bug. If you need any further help, our forum is the best place to ask 😄


fix nits in log msg

Signed-off-by: allencloud<EMAIL_ADDRESS>

Current coverage is 54.86% (diff: 0.00%)

Merging #1306 into master will decrease coverage by 0.17%

@@             master      #1306   diff @@
==========================================
  Files            80         77     -3
  Lines         12488      12390    -98
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           6873       6798    -75
+ Misses         4668       4650    -18
+ Partials        947        942     -5

Powered by Codecov. Last update f3c8fc5...98424e8

LGTM


Flashed Lenovo Ideapad Slim 3i Chromebook (PUJJOTEEN15W) now unable to update

Flashed a Lenovo Ideapad Slim 3i Chromebook (PUJJOTEEN15W) with MrChromebox-4.22.0 (12/22/2023) (full rom/UEFI package).
I went to update with a newer version of the firmware but am not able to do so. Running the MrChromebox script gives the following with option #1 to Install/Update UEFI greyed out.
ChromeOS Device Firmware Utility Script [2024-01-07]
(c) Mr Chromebox<EMAIL_ADDRESS>** Device: ERROR: unknown or unidentifiable device (PUJJOTEEN15W)
** Platform: (unrecognized)
** Fw Type: Full ROM / UEFI
** Fw Ver: MrChromebox-4.22.0 (12/22/2023)
** Fw WP: Disabled
** 1) Install/Update UEFI (Full ROM) Firmware
** [WP] 2) Restore Stock Firmware
** C) Clear UEFI NVRAM
Select a numeric menu option or
R to reboot P to poweroff Q to quit

just pushed a fix, quit and retry in a min or so

With the fix I was able to update the firmware.  Only issue I see is that in the boot options/settings menu that 0 MB RAM is reported (this was happening on the previous version).  Works normally otherwise.  Thank you!

could be the FSP binary needs an update, possible the next release will fix it


feat: append_file incl. all tests [agentskills]

Replacement for discarded first draft: #2207
Adds append_file to Agentskills plugin. Includes unit tests and adapted integration tests.
This PR adds a new method append_file(content: str) to the agent skills.
In testing OD with some file operations, it is a noticable issue for repeated errors by the model (used Gemini Pro 1.5) that adding content to a file often causes exceptions due to an invalid "start" line number for the edit_file command.
It seems very hard for the model to identify or keep track the total number of lines in a file or it is just not good enough in counting (ask an LLM to count the words of its answer and you know what I mean).
This issue can cause extra cost when it shouldn't.
Such an error then looks like this:

An example prompt to demonstrate the use of it could be like:
"Write the numbers 5 to 10 into a new file named test.txt"
A followup prompt could then just say:
"Append the numbers 20 to 25 to the same file."
If it were to use the edit_file command, chances would be high that it would use the wrong line number(s) again.
cc @neubig @li-boxuan
(blocked by https://github.com/OpenDevin/OpenDevin/pull/2085 as well)

It'd be cool if one reviewer could trigger the workflow to start just to see how the tests go, but no merging yet.

Btw why is this blocked by #2085? I think we could finish this PR, merge it, note it down, and do evaluation afterwards.

Btw why is this blocked by #2085? I think we could finish this PR, merge it, note it down, and do evaluation afterwards.
Doing evaluation for every change to prompt might be too costly.

I copied that note about #2085 over, just in case.

Ok, great to see that besides Mac, all the unit and integration tests worked. 🎸

Commit incoming shortly with file_name added as parameter for append-/edit_file methods incl. updated tests...

Also, can someone confirm that this should work without a config.toml present?
TEST_ONLY=true ONLY_TEST_AGENT="CodeActAgent" ./tests/integration/regenerate.sh
For me this errors if no config.toml exists.

Also, can someone confirm that this should work without a config.toml present?

Yes. That's exactly how CI is set up.

Also, can someone confirm that this should work without a config.toml present? TEST_ONLY=true ONLY_TEST_AGENT="CodeActAgent" ./tests/integration/regenerate.sh For me this errors if no config.toml exists.

It works with no toml for me too. Do you have env vars set, like any of the workspace vars? Any chance you see what error it gives you with no toml?

Any chance you see what error it gives you with no toml?

My WSL env doesn't have any OpenDevin vars.
I'll give it a try again without toml later today and try to find out more details.

If you guys are ok with the extra check staying - and avoiding potential WSL issues - then I think this PR is good to go. 🚀

I can run a more comprehensive eval once we fix all the kinks in swe-bench eval and revert when necessary.

Ah, excellent, just overlapped with my reply above. Cheers!

cc @neubig @li-boxuan  ✉ kind invitation for review ✏ 👨 😃

@yufansong please click 🕹 😂


Update profiler.md

Add information on how to enable profiler for other clouds

@juliusl : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@mariohewardt -- Can you review please!

@juliusl : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

#sign-off


Add automatic toml loading

Let's keep this open for when E35 will be released. will do those changes too

func fetchSnapshotHashes(url string) ([]byte, error) {
	resp, err := http.Get(url)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	res, err := io.ReadAll(resp.Body)
	if len(res) == 0 {
		return nil, fmt.Errorf("empty response from %s", url)
	}
	return res, err
}

Btw we already treat empty response as error if their length is 0

oh wait need to resolve comments

added SNAPS_GIT_BRANCH for re-defining


Update the homepage based on user feedback

Update the homepage based on user feedback
Additions

Header slug above the hero news snippets

Changes

Updates the hero images

Testing

gulp build and check out the home page

Review

@KimberlyMunoz
@anselmbradford
@sebworks

Screenshots

Checklist

[x] Changes are limited to a single goal (no scope creep)
[x] Code can be automatically merged (no conflicts)
[x] Code follows the standards laid out in the front end playbook
[x] Passes all existing automated tests
[x] New functions include new tests
[x] New functions are documented (with a description, list of inputs, and expected output)
[x] Placeholder code is flagged
[x] Visually tested in supported browsers and devices
[x] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)

Does  o-well__home still exist? Should it be  o-well__full?

looks great, :+1:

@sebworks thanks for noticing that, I hadn't even noticed. It's fixed now.

:+1:

:+1:


ghc-make can't find package?

Compiling with ghc --make is slow. So I installed ghc-make  on a windows 64 bits machine with ghc version 7.8.3 on it.  Then I did
ghc-make -j4 -threaded -O2 -rtsopts Xls2Prog
and got:
ghc-pkg (for .ghc-make.pkgs)
Error when running Shake build system:

.ghc-make.result
Hulpfunc.o
Hulpfunc.o Hulpfunc.hi
(x86)/Haskell Platform/2<IP_ADDRESS>/lib/containers-<IP_ADDRESS>/Data/Map.hi
Error, file does not exist and no rule available:
(x86)/Haskell Platform/2<IP_ADDRESS>/lib/containers-<IP_ADDRESS>/Data/Map.hi

With ghc --make there is no error message,
Kees

This bug is a bug in the Makefile parser which is fixed in the HEAD version of Shake. I deal with at most one space in a Makefile atom, but if your directory has 2 (or 3 in your case) it breaks the line incorrectly.
I'm hoping to release a new version of Shake shortly (next few days) but am held up by a miscompilation introduced in GHC 7.10.1 RC3 which I need to track down first. In the meantime either wait a little, or install HEAD shake and recompile ghc-make.

OK, Shake has been released now. Do cabal update && cabal install shake && cabal install ghc-make and hopefully it will just work.

No errors anymore.
Thanks !
Kees
Van: Neil Mitchell<EMAIL_ADDRESS>Verzonden: donderdag 26 maart 2015 15:54
Aan: ndmitchell/ghc-make
CC: KeesBleijenberg
Onderwerp: Re: [ghc-make] ghc-make can't find package? (#9)
Closed #9 https://github.com/ndmitchell/ghc-make/issues/9 .
—
Reply to this email directly or view it on GitHub https://github.com/ndmitchell/ghc-make/issues/9#event-265935700 .Beschrijving: Afbeelding verwijderd door afzender.


(#240) Angular CLI Integration Added 2 PrimeNg dependencies

For this new library ..  just use small commit for each task and smal task --> see my own . just package.json has been changed


Change list color

I've changed Check available pages list color to Torch Red in Contributors page. Now it is similar to other pages' Check available pages list color. I hope this PR will be merged. Thank you.

Here is a screenshot of changed color in contributor's page.

Awesome :D


Add coverage check.

On this run the check took 3m38s which makes it the slowest of the CI checks ("tests (nightly, true)" took 3m0s).
I took the simplest approach which was:

run on PR only;
compute coverage both for PR branch and for main;
compare only the total line coverage (ignoring function coverage and region coverage whatever that is, and ignoring individual file reports).

This means we don't need to store anything persistently, but we do need to run the coverage checks twice, which increases the run time.
Obviously we could refine the checks in future, e.g. to include function coverage statistics. We could also look into running on push to main and storing the result persistently, but I'm not very happy with how tket does this (by pushing a file to the magic gh-pages branch).

In the CI job:

Run ./.github/workflows/compare-coverage
line coverage here: 73.40%
line coverage on main: 73.39%

Shouldn't that be equal?
Would it make sense to add an epsilon to the comparison, to avoid any rounding nuisances?

Good spot. There must be some random variation due to sampling I guess. Yes, adding an epsilon sounds like the best way around this.

Run ./.github/workflows/compare-coverage
line coverage here: 73.39%
line coverage on main: 73.42%

O.o

If this gets too annoying with false negatives we can revert.


Quit visual mode when press Escape

After finishing the operation in visual mode, I am used to pressing Escape to quit visual mode, but currently I have to press the same key as the trigger.

Yes It's a good idea, I also want to implement it that way, but It's hard, I have given a suggestion to the Logseq team. When it is possible I will add it right away.


[Question]: RPM packages

Contact Details
Alex Kasko<EMAIL_ADDRESS>What's your question?
My current employer is planning to run an existing application on Babelfish on Amazon Aurora. I was doing local builds of Babelfish to use them in test and for possible non-cloud deployments. I've been familiar with RPM packaging for Fedora (was working for Red Hat in the past) and decided to set up Babelfish RPM builds for Fedora and RHEL.
Fedora Copr is a public build system for community packages. I've set up Babelfish RPM packages builds on Copr for Fedora and for EL 7, 8 and 9. These packages bring some QoL improvements, like systemd integration and adding TDS to preload libs. Perhaps they might be of interest to wider Babelfish community.
If it sounds like a good idea, it is possible to make these packages more "official" - if someone from Babelfish Maintainers would like to create an official Copr account for Babelfish (I've used my existing account from one of my old projects), I can assist with moving the RPM spec repos to Babelfish GitHub org and with Copr builds setup.
On the other hand (if "official" RPMs are undesired), if necessary, I can clearly mark the existing RPMs as "unofficial" (replace in description "Babelfish packages" to "packages based on a source code from Babelfish" etc).
Notes: there are no plans to propose these packages for inclusion into Fedora itself (the nature of "modified PostgreSQL" makes this unrealistic); there are ideas to set up similar DEB packages on Ubuntu PPA, but no immediate plans to do so.
Thanks for making this project open-source!
Version
Initial RPMs are for 2.2.0 (intended to be updated):

BABEL_2_2_0__PG_14_5
BABEL_2_2_0

Extension

babelfishpg-money
babelfishpg-common
babelfishpg-tds
babelfishpg-tsql

Which flavor of Linux are you using
RHEL, EL derivatives,  Fedora.
Code of Conduct

[x] I agree to follow this project's Code of Conduct.

I now build RPMs from modified sources, I've renamed the package from babelfishpg to wiltondb to clearly show that it is not official and moved it to this Copr project. I'll close this issue for now. If the idea of having official Copr builds for upstream Babelfish will ever be desired - I can happily contribute spec files for that.

Why was it called WiltonDB ? Just asking.

Apologies if I am misinterpreting the question, answering "why is it not called Babelfish?":
Main version is Windows one, I cannot call it "Babelfish for Windows", because Babelfish officially does not support Windows. Babelfish codebase relies on a number of assumptions about the toolchain (for example: 128-bit integer support, variable-length struct fields, extern variables visibility) that are incompatible with MSVC compiler. Windows porting patches applied to Babelfish are numerous and not trivial, if WiltonDB misbehaves or crashes when running on Windows (example of such crash), it does not necessary mean that Babelfish running on Linux will have the same problem. Linux packages are supplementary and are built from the same modified sources as Windows one, so also named WiltonDB to clearly show that these packages are not "vanilla Babelfish" (like docker-babelfishpg), for example this patch is included (because it is important for PyODBC usage) that is currently not present in any released Babelfish version. The name itself does not have any intrinsic meaning - just a geographical name.

Thanks, I was just curious. Is that Wilton, Salisbury, UK ?

It is initially named after the Wilton street in central London, overall it is a relatively popular geographical name.

Curiosity satisfied.


HDFS-16108. Fix incorrect log placeholders used in JournalNodeSyncer

Thanks @tomscut for the review.
@aajisaka could you please take a look?
Thanks

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 13s

trunk passed

+1 :green_heart:
compile
1m 25s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
1m 15s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  0s

trunk passed

+1 :green_heart:
mvnsite
1m 24s

trunk passed

+1 :green_heart:
javadoc
0m 55s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 26s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m 16s

trunk passed

+1 :green_heart:
shadedclient
18m 58s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 15s

the patch passed

+1 :green_heart:
compile
1m 18s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
1m 18s

the patch passed

+1 :green_heart:
compile
1m 10s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
1m 10s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 56s

the patch passed

+1 :green_heart:
mvnsite
1m 16s

the patch passed

+1 :green_heart:
javadoc
0m 48s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 20s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m 25s

the patch passed

+1 :green_heart:
shadedclient
18m 54s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
337m 18s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

429m 17s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.TestDecommissioningStatus

hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor

hadoop.hdfs.TestDFSShell

hadoop.hdfs.server.namenode.ha.TestEditLogTailer

hadoop.hdfs.server.namenode.ha.TestBootstrapStandby

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3169/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3169

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 9916232dc298 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d1c06047d5dece5fb94eceddb4af5bbdee9499c6

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3169/1/testReport/

Max. process+thread count
2026 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3169/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
30m 32s

trunk passed

+1 :green_heart:
compile
1m 23s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
1m 16s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  2s

trunk passed

+1 :green_heart:
mvnsite
1m 23s

trunk passed

+1 :green_heart:
javadoc
0m 55s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 30s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m  6s

trunk passed

+1 :green_heart:
shadedclient
16m  9s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 12s

the patch passed

+1 :green_heart:
compile
1m 16s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
1m 16s

the patch passed

+1 :green_heart:
compile
1m  9s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
1m  9s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 53s

the patch passed

+1 :green_heart:
mvnsite
1m 14s

the patch passed

+1 :green_heart:
javadoc
0m 46s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 21s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
3m 16s

the patch passed

+1 :green_heart:
shadedclient
16m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
239m  8s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 43s

The patch does not generate ASF License warnings.

323m 23s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3169/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3169

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 8424f1079d4a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 507a47a17163d5759a5024d4c28719aa199eac14

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3169/2/testReport/

Max. process+thread count
3033 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3169/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@ferhui could you please take a look? Thanks

@virajjasani Thanks for contribution. @aajisaka @tomscut Thanks for review.
Merged to trunk.


style: add primary colors scheme

We can't test themes properly because Jest can't render styles. We can think in another alternative, maybe an integration test.