multiple lines autoclosing at once creates too many closings

To replicate this error you need to have multiple lines being edited at once. Open a new tag and when you type in > to close the tag it will create and extra closing tag on each opened tag except for the very last one.
Here is a couple of screenshots when I might try typing in several list items at once. you can see on each list item after I close the tag it generates an extra </li> that is not needed

I also did one with more to show that the number of closes does not scale up with the number of tags opened at once. you just need multiple to be opened at once.

Here is what happens when I do this with a block element and not an inline one

Dupe of #76

@miscbits With 0.22.0 I changed the method of detecting the end of a tag, which made this a bit better. however you lose the multiple cursors, which isn't perfect.
I'll refine it a bit more in future releases

awesome. I'll take a look at the changes if you dont mind and see if I cant figure something out as well!


ResponseDelegate can invoke didReceiveBodyPart before the previous run's future returned

I've added a PR #273 to verify this

Yes, right now, the response delegate may be called multiple times. The backpressure itself does seem to work (it won't give you arbitrary amounts of data until you processed it) but there's no guarantee it'll only be invoked once.
The code backs this up
    func channelRead(context: ChannelHandlerContext, data: NIOAny) {
        let response = self.unwrapInboundIn(data)
        switch response {
        case .head(let head):
            [...]
        case .body(let body):
            switch self.state {
            case .redirected:
                break
            default:
                self.state = .body
                self.mayRead = false
                self.callOutToDelegate(value: body, channelEventLoop: context.eventLoop, self.delegate.didReceiveBodyPart)
                    .whenComplete { result in
                        self.handleBackpressureResult(context: context, result: result)
                    }
            }
        case .end:
            [...]
    }

    public func read(context: ChannelHandlerContext) {
        if self.mayRead {
            self.pendingRead = false
            context.read()
        } else {
            self.pendingRead = true
        }
    }

As we can see, AHC will send everything to the delegate straight away that comes through channelRead. channelRead however can be invoked multiple times even if you're not reading. For example because:

a ChannelHandler in front just invokes it multiple times
we receive EOF (when NIO will drain the socket buffer and deliver the remaining data)
maxMessagesPerRead != 1

AHC is meant for end users so I would agree that it should invoke the delegate again only after having seen the previous future succeed. That's by far the most useful model for users who may say want to write the received body to disk or so.
[SwiftNIO itself couldn't reasonably make such a guarantee because the user (ie the ChannelHandlers are in charge of calling channelRead). Sure, NIO could in theory buffer before/after each ChannelHandler but that would add a lot of memory bloat that the user can't control.]

CC @artemredkin . Should we make this a 1.2.0 release blocker or are we "happy" with it given that it was always broken?

I think we should schedule it for 1.2.1 milestone since it was already broken.

The test in #273 also shows that the HTTPClient finishes before all the ResponseDelegate tasks are finished. Would this be considered a different bug?

The test in #273 also shows that the HTTPClient finishes before all the ResponseDelegate tasks are finished. Would this be considered a different bug?

This would be fixed by fixing the above bug, but this could possibly be resolved without fixing the above. I believe it is a more serious issue than the original bug filed. Should this get its own issue? It has been sitting around for 6 months now.

The test in #273 also shows that the HTTPClient finishes before all the ResponseDelegate tasks are finished. Would this be considered a different bug?

This would be fixed by fixing the above bug, but this could possibly be resolved without fixing the above. I believe it is a more serious issue than the original bug filed. Should this get its own issue? It has been sitting around for 6 months now.


Revisit operations when grouped

Ideally we will have a separate test file for testing how each of them works when grouped, removing all of .group calls from data_frame_test.exs.
Group handling

[x] group_by
[x] ungroup
[x] summarize

Behaves different under group:

[x] arrange (in dplyr the group behaviour is opt-in)
[x] arrange_with
[x] collect (document that collecting a grouped data frame should return a grouped data frame)
[x] distinct
[x] head
[x] mutate
[x] mutate_with
[x] rename (rename grouped columns)
[x] rename_with (rename grouped columns)
[x] sample (sample 5 of each group)
[x] select (raise if dropping grouped columns) => should raise? today is no-op if grouped is passed
[ ] slice
[x] tail
[x] ~take~ (renamed to slice)
[x] filter_with

No changes under group

[x] ~filter~ (is now only filter_with)
[x] n_columns
[x] n_rows
[x] pull
[x] shape
[x] to_* (a small test for some of them is enough)

To discuss

[ ] dummies
[ ] pivot_longer
[ ] pitot_wider
[ ] concat_rows (what if one side has the grouping and the other does not?)
[ ] join (what if one side has the grouping and the other does not?)

I think slice/2 won't work with ranges for a grouped dataframe, because we can't know the size of each group beforehand. I think we could only support slice/2 with row indices. WDYT?

@josevalim yeah, makes sense. :+1:
I'm going to try that. My only concern is that Rust ranges don't accept steps. But I can validate that in the backend.

I'm investigating the rest of the operations (the ones from "To discuss" section) and I will follow the behaviour we see in dplyr, if that is OK for you.
For pivot_longer and pivot_wider the groups are kept if they are not used in the pivoting operation. For example, if we have a group that is used as a column to pivot_longer, that group is not going to appear in the resultant dataframe. I have to fix our code to follow this behaviour.

The join operations described in https://dplyr.tidyverse.org/articles/two-table.html don't specify what happens with groups, but I found that the groups are kept if they are in the left-hand side of the operation.
So, for example, if we have two dataframes:
df1 = Explorer.DataFrame.new(x: [1, 2], y: [2, 1])

df2 = Explorer.DataFrame.new(x: [3, 1], a: [10, 10], b: ["a", "a"])
Explorer.DataFrame.inner_join(df1, df2)

If we add a group in df1, this group is going to be present in the resultant dataframe.
Buf if we add a group in the df2, it is not going to be in the joined DF.
Do you think this behaviour is fine?

For left join it makes sense to keep one side but not the other. Then inner_join makes sense to mirror left_join. So I agree with this behaviour.

For concat_rows, I could only find the union/union_all operations that are similar in dplyr: https://dplyr.tidyverse.org/reference/setops.html
They will keep only the groups of the first dataframe (or the left-hand side df), and ignore the rest.
I think this is fine, and I'm going to document/test that.

Closing this, since all the functions are fixed and documented. :tada:
If we have more that are not in this list, we should create individual issues. :)


Deferred components should not load on non-UI thread

When testing with flutter gallery, sometimes, the calls are made on a non-UI thread. This should only be done on UI thread.

https://github.com/flutter/engine/pull/25004 should resolve this


Some of the check lists in the MD files need cleanup

The converter I created seems to have made some minor mistakes, this is a reminder for me to get these lists cleaned up.
E.g. Child safety

Actually I have changed some MD file contents.
For example feature_color
At first value is hex so I couldn't read hex value with graphql
So I tried to get data and as a result I wrapped it with quotes

Also youtube video is not loading as proper width and height.
I think it is depends on markdown files.
If you changed it then I can fix some css styles.

Leave for M5


Add new talk to "Talks" page

Add Summayah Ahmed's talk to talks page - from Droid Con:
https://www.droidcon.com/media-detail?video=481205520

completed


Screenshot is of the Windows 10 lockscreen and not the failed test

Are you requesting a feature or reporting a bug?
Bug
What is the current behavior?
If the screenshot is taken while the PC is locked, then the screenshot will be of the Windows lockscreen.
What is the expected behavior?
The screenshot should be of the failed test in the browser.
How would you reproduce the current behavior (if this is a bug)?

Set Windows to lock the PC after 1 minute
Run a failing test that takes longer than a minute and that will take screenshots

Specify your

operating system: Windows 10
testcafe version: 0.20.4
node.js version: 8.9.4

Hi @row1! Because Windows doesn't provide a reliable API to take a screenshot of a specific window, so in general case TestCafe have to capture screenshot of the desktop and cut a image of the window from it. Of course if you lock you computer, you will get a screenshot of the lockscreen instead.
TestCafe uses this general approach for IE and Edge. Fortunately, Chrome and Firefox provide the possibility to capture screenshots of a browser window using their debug protocols, so the lockscreen shouldn't cause troubles with them.
TL&DR: If you experience this with IE and Edge, it's a Windows technical limitation, and I'm afraid you  have to disable your lockscreen. If it happens in Firefox or Chrome, it's a bug, so please provide me the TestCafe command that you used to run tests and the corresponding test report.

Sorry, I should have mentioned that i have been using Chrome.
I see that v0.21.0 has just been released (thanks), so I will try upgrading to that first. I will also see if using headless Chrome makes a difference. Will report back next week.
Thanks

You can close this. I have upgraded from 0.20.4 to 0.21.0 and it seems to be working fine now.

@row1, thanks for feedback!


Ignore namespace for loaded cs files in Jupyter Notebook

I tried to reuse the entities defined in a .NET solution into a Jupyter Notebook. The entities are part of a namespace in my solution, so I cannot remove the namespace without impacting the whole solution.
It would be great if the existing namespace is simply ignored when the file is loaded in Jupyter Notebook.
MyNotebook.ipynb
#load "Person.cs"
Person.cs
namespace Entities
{
  public class Person { public string Name { get; set; } }
}

Related: https://github.com/dotnet/csharplang/issues/3117

I know this is old, but I don't see how this is completed? I would still love this feature such they I can use a notebook next to an existing .net solution and not need to remove all namespaces. for me the obvious fix would be to simply ignore namespaces when using #import


Implementing nested type constraints

I'd love to hear your advice on how to implement nested type constraints.
Introduction
When I see the following spec
// Semigroup
concat :: Semigroup s => (s, s) → s

I automatically define the following 2 things (using pseudo code and the flow syntax in order to emphasize the types)
// Semigroup.js
export interface Semigroup<S> {
  concat(a: S, b: S): S
}

export function concat<S>(dictSemigroup: Semigroup<S>, a: S, b: S): S {
  return dictSemigroup.concat(a, b)
}

Here Semigroup<S> mimic a type class while dictSemigroup represent the Semigroup s => type contraint.
An example
const StringSemigroup = {
  concat(a: string, b: string): string {
    return a + b
  }
}

import { concat } from './Semigroup'
concat(StringSemigroup, 's1', 's2') // => 's1s2'

The problem
The problem comes when there are nested type contraints. Let's define a Semigroup instance for Maybe
// I can't implement this
const MaybeSemigroup = {
  concat<S>(a: Maybe<S>, b: Maybe<S>): Maybe<S> {
    if (isNothing(a) || isNothing(b)) {
      return Nothing
    }
    return Just(concat(?, a, b)) // <= here's the problem; I don't have an instance of Semigroup for S
  }
}

How could I implement MaybeSemigroup?
Now I'm doing something like this but I'm not entirely satisfied, are there other options?
function getMaybeSemigroup<S>(dictSemigroup: Semigroup<S>): Semigroup<Maybe<S>> {
  return concat<S>(a: Maybe<S>, b: Maybe<S>): Maybe<S> {
    if (isNothing(a) || isNothing(b)) {
      return Nothing
    }
    return Just(concat(dictSemigroup, a, b))
  }
}

Interesting issue. We discussed it just yesterday in Fantasy Land's chat: https://gitter.im/fantasyland/fantasy-land?at=57b335271a7d02075684055b .
Yes, you need to have type dictionary of inner type in order to implement outer type's concat in Static Land. This is inherent limitation of Static Land compared to Fantasy Land https://github.com/rpominov/static-land#cons . We have to pass around type objects for any generic code.

Ugh, seems like Gitter swallows a huge part of messages in a link like that. It works better if you just open chat (https://gitter.im/fantasyland/fantasy-land) and scroll up. Or maybe it's only me who sees the bug, and first link will also work.

Is this fine for the spec? I'm implementing a These library inspired by @joneshf's example in Elm). If I implement a makeChain function that takes a type dictionary for a monoid, am I OK to be a static-land monad?

I guess no one can answer for sure what is fine, we're all still exploring how this stuff fits in JS. But from strict static-land spec perspective: if a type dictionary obeys all laws for a particular algebra it's a correct static-land type that implements that algebra.
So if your makeChain function takes a type dictionary for a Monoid, and returns a type dictionary for Chain, for example, the only requirements for the result type dictionary is to obey Chain laws.

As a side note, after using that technique for 9-10 days I can say I actually like it. At first seems a bit awkward (well, it is..) but unlike some Haskell/PureScript code where it's not always easy to guess which instance is involved, passing those dictionaries it's very explicit and easy to reason about, especially when you re-read your code after a few days. So I... "made a virtue of necessity"

Yeah I hadn't thought of returning a dictionary. I actually really like the fact that means one is explicit about fixing one of the type variables in my case

El 27/08/2016, a las 16:29, Giulio Canti<EMAIL_ADDRESS>escribió:
As a side note, after using that technique for 9-10 days I can say I actually like it. At first seems a bit awkward (well, it is..) but unlike some Haskell/PureScript code where it's not always easy to guess which instance is involved, passing those dictionaries it's very explicit and easy to reason about, especially when you re-read your code after a few days. So I... "made a virtue of necessity"
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

@gcanti You could also be lazy. Instead of saying, "Give me a type dict and two maybes and I'll return to you a new maybe." You could say, "Give me two maybes, and I'll return to you a function that needs a type dict to get a new maybe."
I'm not using flow since its syntax is incredibly bad.
const MaybeSemigroup = {
  concat(a, b) {
    return function(dictSemigroup) {
      return Maybe.ap(Maybe.map(x => y => concat(dictSemigroup, x, y), a), b);
    }
  }
}

You just push the responsibility to the caller rather than dealing with it yourself. Of course, that's no longer a Semigroup since
concat : (Maybe a, Maybe a) -> Semigroup a -> Maybe a

But you could change the type to be
concat : (Semigroup a -> Maybe a, Semigroup a -> Maybe a) -> Semigroup a -> Maybe a

const MaybeSemigroup = {
  concat(fa, fb) {
    return function(dictSemigroup) {
      const a = fa(dictSemigroup);
      const b = fb(dictSemigroup);
      return Maybe.ap(Maybe.map(x => y => concat(dictSemigroup, x, y), a), b);
    }
  }
}

This is slightly different from the original problem. But, if you look closely, that's a specific instance of Kleisli m a b. In this instance we have Kleisli Maybe (Semigroup a) a.
Also, I feel like you could write a kleisli semigroup once and for all m. It would only be one kind of Semigroup for a given m, but it seems like it would be principled.
Just some food for thought. :hamburger:

Btw, what is a function that takes a type dictionary and returns another type dictionary? Is this higher kinded static-land type? 😮

Does it need a special name?

Maybe, if that become a common pattern like "Higher-Order Components" in React community. But it just seemed interesting to me to make that analogy.

Thanks @joneshf, very interesting.
My main goal here was to come up with a "mechanical" and straightforward rule aimed to translate a PureScript signature into a FlowScript (or JavaScript) one, such that all the implementations are consistent. For example, here's a translation almost word by word
-- PuresScript
instance semigroupMaybe :: Semigroup a => Semigroup (Maybe a) where
  append Nothing y = y
  append x Nothing = x
  append (Just x) (Just y) = Just (x <> y)

to
// flow-static-land AKA FlowScript
const semigroupMaybe: (semigroupA: Semigroup<A>) => Semigroup<Maybe<A>> = semigroupA => ({
  concat(x, y) {
    if (prj(x) == null || prj(y) == null) {
      return Nothing
    }
    return of(semigroupA.concat(prj(x), prj(y)))
  }
})

where type constraints translates nicely to dict arguments

Ah, then yeah, that's much better :).

This is a great discussion, and please keep it going, but I'll close the issue since it seems resolved.


ISPN-11184 Infinispan MicroProfile metrics are registered to the wron…

…g scope
https://issues.redhat.com/browse/ISPN-11184
Needs backport to 10.1.x

LGTM

I'll merge it, but I've basically wiped out metrics and reimplemented it in a different way :). PR to come today. there was a lot more wrong about it than the scope

I've backported to 10.1.x also, @rhusar


Integrate code coverage scanning

As part of the ongoing support for versions, etc we'd like to ensure we don't break APIs! One of the ways to do this, in line with other clients is to integrate codecov.
I'll happily do the integration - but we need to connect this to codecov - via their interface as well.

It's connected 👍

See #1036.

Seeing coveralls integration I think we can close this. Coverage is coverage.


STT DEMO KWS -- Inconsistencies with punctuation: Some messages end in a period and some do not

Leonid,
On the STT demo page (via this link -->https://speech-to-text-demo-lr.mybluemix.net/), depending on the scenario, some times I get a message.
Why do some messages end with a period and some do not? See list below:
Upsampling not allowed because recognition accuracy will suffer. Sampling rate of at least 16000 required. 	(Sentence ends with a period.)
Only WAV or FLAC or Opus files can be transcribed, please try another file format				(Sentence does not end with a period.)
Notice: browers do not support playing FLAC audio, so no audio will accompany the transcription			(Sentence does not end with a period.)
Please let me know if you have any questions or comments.
Alexandra

irrelevant


0.54

New Platforms

Added mqtt_statestream component (@mw-white - #9286) (mqtt_statestream docs) (new-platform)
Add support for Todoist platform (@Jay2645 - #9236) (calendar.todoist docs) (new-platform)
Basic MQTT vacuum support (@johnboiles - #9386) (vacuum docs) (vacuum.mqtt docs) (new-platform)
Added Zyxel Keenetic NDMS2 based routers support for device tracking (@foxel - #9315) (device_tracker.keenetic_ndms2 docs) (new-platform)
DoorBird Component (@Klikini - #9281) (doorbird docs) (binary_sensor.doorbird docs) (camera.doorbird docs) (switch.doorbird docs) (new-platform)
Telnet switch (@multiholle - #8913) (switch.telnet docs) (new-platform)
Abode services, events, lights, cameras, automations, quick actions. (@1091741+MisterWil - #9310) (abode docs) (alarm_control_panel.abode docs) (binary_sensor.abode docs) (camera.abode docs) (cover.abode docs) (light.abode docs) (lock.abode docs) (switch.abode docs) (new-platform)
Added satel_integra alarm panel and binary sensor platform (@c-soft - #9336) (satel_integra docs) (alarm_control_panel.satel_integra docs) (binary_sensor.satel_integra docs) (new-platform)
Worx Landroid sensor (@nilzen - #9416) (worx-landroid docs) (sensor.worx-landroid docs) (new-platform)

Breaking Changes

Added more devices and types to onewire (@tcastberg - #9404) (sensor.onewire docs) (breaking change)
Rename xiaomi #9425 (@danielhiversen - #9426) (breaking change)
Refactor Swiss Public Transport sensor (@fabaff - #9129) (sensor.swiss_public_transport docs) (breaking change)
fitbit fixes (@happyleavesaoc - #9460) (sensor.fitbit docs) (breaking change)

All changes

Add HTTP Basic auth to RESTful Switch (@amelchio - #9162) (switch.rest docs)
Bump pywebpush and pyJWT versions (@fanaticDavid - #9355) (notify.html5 docs)
Add /usr/sbin to PATH (@morberg - #9364)
Add polling interval service and setting available through zwave node entity panel (@turbokongen - #9056) (zwave docs) (config.zwave docs)
Round off probability to 2 decimals. (@arsaboo - #9365) (binary_sensor.bayesian docs)
Added mqtt_statestream component (@mw-white - #9286) (mqtt_statestream docs) (new-platform)
typo in waypoint import topic preventing waypoint import (@diplix - #9338) (device_tracker.owntracks docs)
Cleanup and simplitfy the async state update (@pvizeli - #9390)
Update cloud auth (@balloob - #9357) (cloud docs) (http docs) (new-platform)
Allow multiple observations of same entity (@jlmcgehee21 - #9391) (binary_sensor.bayesian docs)
Break up Alexa per functionality (@balloob - #9400) (alexa docs) (alexa.const docs) (alexa.flash_briefings docs) (alexa.intent docs) (new-platform)
Upgrade psutil to 5.3.1 (@fabaff - #9403) (sensor.systemmonitor docs)
fix mopar sensor (@happyleavesaoc - #9389) (sensor.mopar docs)
Support specifying no Apple TVs (@postlund - #9394) (apple_tv docs)
Update netdisco to 1.2.0 (@balloob - #9408)
components/xiaomi: Add initial discovery using NetDisco. (@pfalcon - #9283) (xiaomi docs)
Fix copy&paste mistake (@tinloaf - #9378)
Added log-file command line flag (@TD22057 - #9422)
Add LC_CTYPE to environment variables in macOS (@morberg - #9227)
Added more devices and types to onewire (@tcastberg - #9404) (sensor.onewire docs) (breaking change)
Bump uvcclient to 0.10.1 to work with beta NVR releases (@antonym - #9423) (camera.uvc docs)
full RGB support for users of tradfri GW (@31916694+spektren - #9411) (light.tradfri docs)
Add support for Todoist platform (@Jay2645 - #9236) (calendar.todoist docs) (new-platform)
Fix artwork bug in Apple TV (@postlund - #9415) (media_player.apple_tv docs)
Bump python-mirobo for improved device support and introduce API changes. (@syssi - #9424) (light.xiaomi_philipslight docs) (vacuum.xiaomi docs)
MPD small improvements (@24255269+pdanilew - #9301) (media_player.mpd docs)
update broadlink.py to add support for MP1 switch (@giangvo - #9222) (switch.broadlink docs)
Fix displaying of friendly_name for light template component (@rollbrettler - #9413) (light.template docs)
Improve Python script (@doudz - #9417) (python_script docs)
Add manual alarm_control_panel pending time per state (@snjoetw - #9264) (alarm_control_panel.manual docs)
Rename xiaomi #9425 (@danielhiversen - #9426) (breaking change)
Bump version of aioautomatic (@armills - #9435) (device_tracker.automatic docs)
Fix russound_rio for python 3.4 (@wickerwaka - #9428) (media_player.russound_rio docs)
Basic MQTT vacuum support (@johnboiles - #9386) (vacuum docs) (vacuum.mqtt docs) (new-platform)
Optimaze vacuum mqtt platform (@pvizeli - #9439) (vacuum.mqtt docs)
Polymer access to log file broken when using new log file command line (@TD22057 - #9437) (api docs)
Allow empty hostnames when detecting devices with the aruba device_tracker. (@michaelarnauts - #9440) (device_tracker.aruba docs)
mqtt_statestream: Update to append 'state' to topic for future use with mqtt discovery (@mw-white - #9446) (mqtt_statestream docs)
Fix for DTE Energy Bridge returning the wrong units from time to time (@kylehendricks - #9246) (sensor.dte_energy_bridge docs)
Refactor Swiss Public Transport sensor (@fabaff - #9129) (sensor.swiss_public_transport docs) (breaking change)
Added Zyxel Keenetic NDMS2 based routers support for device tracking (@foxel - #9315) (device_tracker.keenetic_ndms2 docs) (new-platform)
Add city/state/country options and fix bugs for airvisual (@bachya - #9436) (sensor.airvisual docs)
Fix emulated hue warning message (@astone123 - #9452) (emulated_hue docs)
Alexa smart home native support (@pvizeli - #9443) (alexa.smart_home docs) (new-platform)
fix for Twitter notifications without media (@MikeChristianson - #9448)
Load WebComponent polyfill on header. (@fanthos - #9438)
Fix issue 5728: Emulated Hue UPnP crashes on special characters. (@SqyD - #9453)
Ios notify camera fix (@rbflurry - #9427)
Adds MQTT Fan Discovery (@hufman - #9463)
Upgrade sqlalchemy to 1.1.14 (@fabaff - #9458)
Upgrade uber_rides to 0.6.0 (@fabaff - #9457) (sensor.uber docs)
Upgrade youtube_dl to 2017.9.15 (@fabaff - #9456)
DoorBird Component (@Klikini - #9281) (doorbird docs) (binary_sensor.doorbird docs) (camera.doorbird docs) (switch.doorbird docs) (new-platform)
Fix recorder does not vacuum SQLite DB on purge (@milanvo - #9469)
fitbit fixes (@happyleavesaoc - #9460) (sensor.fitbit docs) (breaking change)
Fix typo in services.yaml (@mika - #9475)
Upgrade pyasn1 to 0.3.5 and pyasn1-modules to 0.1.4 (@fabaff - #9474) (notify.xmpp docs)
Xiaomi Gateway: Allow static configuration of a gateway without discovery (@syssi - #9464) (xiaomi_aqara docs)
Added support for the DTE Energy Bridge v2 (@452469+Blender3D - #9431) (sensor.dte_energy_bridge docs)
Telnet switch (@multiholle - #8913) (switch.telnet docs) (new-platform)
Abode services, events, lights, cameras, automations, quick actions. (@1091741+MisterWil - #9310) (abode docs) (alarm_control_panel.abode docs) (binary_sensor.abode docs) (camera.abode docs) (cover.abode docs) (light.abode docs) (lock.abode docs) (switch.abode docs) (new-platform)
Added satel_integra alarm panel and binary sensor platform (@c-soft - #9336) (satel_integra docs) (alarm_control_panel.satel_integra docs) (binary_sensor.satel_integra docs) (new-platform)
Worx Landroid sensor (@nilzen - #9416) (worx-landroid docs) (sensor.worx-landroid docs) (new-platform)
Fix universal media_player mute (@colindunn - #9462) (media_player.universal docs)
Small improvement of KNX Covers (@Julius2342 - #9476)
Upgrade async_timeout to 1.4.0 (@fabaff - #9488)
Bump version of abodepy (@1091741+MisterWil - #9491) (abode docs)
Upgrade coinmarketcap to 4.1.1 (@fabaff - #9490) (sensor.coinmarketcap docs)
Upgrade blockchain to 1.4.0 (@fabaff - #9489)
Solve Recorder component failing when using Axis component (@Kane610 - #9293) (axis docs) (camera.axis docs)
switch to pypi for xiaomi gw (@danielhiversen - #9498) (xiaomi_aqara docs)
renamed add_devices to async_add_devices according to hass naming scheme (@Julius2342 - #9485) (binary_sensor.knx docs) (climate.knx docs) (light.knx docs) (sensor.knx docs) (switch.knx docs)
Revert "renamed add_devices to async_add_devices according to hass naming scheme (#9485)" (@pvizeli - #9503) (alarm_control_panel.spc docs) (binary_sensor.knx docs) (binary_sensor.spc docs) (climate.knx docs) (light.knx docs) (sensor.citybikes docs) (sensor.knx docs) (sensor.worxlandroid docs) (switch.knx docs)
LIFX: fix multi-zone color restore after effects (@amelchio - #9492) (light.lifx docs)
renamed add_devices to async_add_devices according to hass naming scheme (second try after failed #9485) (@Julius2342 - #9505) (alarm_control_panel.spc docs) (binary_sensor.knx docs) (binary_sensor.spc docs) (climate.knx docs) (light.knx docs) (sensor.citybikes docs) (sensor.knx docs) (sensor.worxlandroid docs) (switch.knx docs)
Xiaomi pycryptodome (@danielhiversen - #9511) (xiaomi_aqara docs)
Bumped pyhomematic, additional device support (@danielperna84 - #9506) (homematic docs) (sensor.homematic docs)
abode: Bump abodepy dependency to 0.11.7 (@vickyg3 - #9504) (abode docs)
abode: Set device_type in state attributes (@vickyg3 - #9515) (abode docs)
update xiaomi aqara lib (@danielhiversen - #9520) (xiaomi_aqara docs)
added services.yaml integration for input_boolean (@skalavala - #9519) (input_boolean docs)
Add reload service to python_script (@doudz - #9512) (python_script docs)
MQTT Binary Sensor - Add availability_topic for online/offline status (@30442019+marthoc - #9507) (binary_sensor.mqtt docs)
Fix typo within cover/knx https://github.com/XKNX/xknx/issues/64 (@Julius2342 - #9527) (cover.knx docs)
LIFX: improve performance of setting multi-zone lights to a single color (@amelchio - #9526) (light.lifx docs)

@balloob  #9427 Will break actionable notifications to IOS devices sent by scripts/automations if the category identifier is not lowercase.
service: notify.ios_<your_device_id_here>
data:
  message: Motion detected in the Living Room
  data:
    push:
      category: CAMERA # this needs be lowercase. after 9427 is merged
    entity_id: camera.demo_camera

@rbflurry Labeled your PR as Breaking Change. Either here or in that PR, could you write up a paragraph explaining the change suitable to include in the release notes? Thanks!

0.53 introduced a bug with the HomeMatic component (https://github.com/home-assistant/home-assistant/issues/9360) which breaks the usage of the variables available on the HomeMatic hub. This has already broken the setups of a few people in the forum. I wonder if it should be mentioned in the release blogpost that people still running HASS < 0.53 should not update until the bug is fixed IF they are using the variables. Any opinions on this?

@danielperna84 can you write up a paragraph and I'll include it in the release notes.

Sure @balloob :
Version 0.53 introduced a bug in the HomeMatic component which breaks the usage of system variables on the CCU / Homegear. This has not been fixed yet. If you are using these variables in your automations you should skip updates until the bug has been resolved. Upgrading is safe however if you only interact with devices.


pyarrow-0.9.0: OS X package missing from PyPi

Hi, it seems that the latest 0.9.0 release for OS X hasn't been published in PyPi. I hope OS X is still a supported platform?
https://pypi.python.org/pypi/pyarrow/0.9.0
Thanks in advance!

We're working on it. Please wait for a while!
#1810

@pyykkis we would appreciate help with packaging -- this project is supported by volunteers

Thanks @kou, I didn't notice there was an existing issue as closed issues are not shown by default by github issue filter.
Thanks @wesm, I forked the repo and looked around a bit and stumbled up to
https://github.com/apache/arrow-dist/pull/25. Looks like @xhochy has pretty much all set. I'm happy to help if there's anything left to do.

pyarrow==0.9.0.post1 is now on PyPI


[WIP] [Utils] Add support for Ninja in Update Checkout script

Add support for Ninja in Update checkout script and checkout tag v1.7.2

@swift-ci Please smoke test

Already done by another PR.


[BUG]fix zkBookieRackAffinityMapping bug to support for bookkeeper dnsResolver

Motivation
When using Region/Rack aware placement policy for pulsar managed ledger, we should also use those policy for bookkeeper and bookkeeper.conf should add the follow configuration. Otherwise, it the placement policy won't work for bookkeeper auto recovery.
ensemblePlacementPolicy=org.apache.bookkeeper.client.RegionAwareEnsemblePlacementPolicy
reppDnsResolverClass=org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping

However, if we configured change root for Zookeeper in bookkeeper.conf, eg.
zkLedgersRootPath=/test-v1/ledgers
zkServers=localhost:2181

/test-v1 is the change root for zookeeper.
The zkBookieRackAffinityMapping resolver will be invalid, and all resolved result will be null.
The reason is that zkBookieRackAffinityMapping initiate Zookeeper Client according to zkServers configured in bookkeeper.conf, and doesn't take change root into account. Thus it lead to get bookies placement policy from wrong place  , which is /bookies and not the correct /test-v1/bookies.
Changes

Check whether configured Zookeeper change root in bookkeeper.conf before initiate Zookeeper Client. If it configured, use the change root path to initiate Zookeeper Client.

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks


Changed my role to "project mommy"

I'd appreciate it if these changes were applied to master, develop. Thanks <3

master abf3eb0778d56d1013d764773dd4335dc566fa74
develop ff58846702fa29af84ff418580cacda8eb8ed6a0


Add a command to list functions in a function app. Closes #187

@ahmelsayed is there a plan on getting this finished up per your comment in #187 ?

Oops, I forgot about it. I don't normally nag myself for my own PRs 😄. Will get it in today.

Thankyou!


Is it possible to make pagination only for the initially received array?

So, on the request to some "index" page (which contains more pagination views in the frontend), I fill different data into different arrays.
For some of those, I want the pagination to work as it is - so, to make ajax request to the simple method in the backend, which returns new results.
But, on the other side, I have data which constantly changes, and every request to the method in the backend returns completely different results. What I want to succeed here, is to make pagination work only for the data that was received on the first request - so that there is no more GET/AJAX requests.
Is that possible? Thanks in advance.

@peric We only accept bug reports and pull requests on GitHub. If you have a support question about how to use kaminari, please ask on StackOverflow.
This request, a.k.a cursor-based pagination, has been request a couple of times (#733, #562) but we don't currently provide such a pagination mechanism.


JS Publishing

In this PR we configure NPM publishing for the spine-js-client package.
The package is published on each master merge, together with all the other artifacts.
Also in this PR:

Spine and Google proto definitions are extracted into a separate Node module and also published;
the Node modules are copied into an isolated directory for build and publish (this is required in order to prepare the package descriptor before passing to npm);
JS-related Gradle declarations are moved into a separate file - js.gradle; we may want to extract that into a standalone Gradle plugin someday;
JS source code now uses 4-space margin instead of 2-space;
the library now supports CORS requests.

@armiol, PTAL.

@armiol, PTAL again.


TSCBasic: enable proper rendering on Windows

This restores the behaviour of rendering on Windows after a very long
time.  However, since TERM is not an environment variable on Windows,
simply elide the check for that.  This finally gives us proper rendering
on Windows.

This depends on https://github.com/apple/swift-package-manager/pull/4192 to get proper rendering.

@swift-ci please test

I think that is part of the problem, I don't have a good way to test this in a test case.  The behaviour is not something that is entirely in our control.  It requires synchronization between a test application, OS, llbuild, and s-p-m.  On this own this is just undoing the blanket "do not support rendering terminals".  This was a workaround for the underlying problem which has now been resolved.


(SignatureError) The transaction is not signed, but tx worked?

Environment information

OS: linux
Python 3.10.12

$ ape --version
0.7.0

Installed Plugins
  alchemy      0.7.0
  arbitrum     0.7.0
  ens          0.7.0
  etherscan    0.7.0
  foundry      0.7.0
  hardhat      0.7.0
  infura       0.7.0
  solidity     0.7.0
  template     0.7.0
  tokens       0.7.0
  vyper        0.7.0

Contents of your ape-config.yaml

  - name: solidity
  - name: vyper
  - name: foundry
  - name: infura
  - name: tokens
  - name: etherscan
  - name: arbitrum

# require OpenZepplin Contracts
dependencies:
  - name: openzeppelin
    github: OpenZeppelin/openzeppelin-contracts
    version: 4.8.1
  - name: openzeppelin-upgradeable
    github: OpenZeppelin/openzeppelin-contracts-upgradeable
    version: 4.8.1
  - name: vault
    github: yearn/yearn-vaults
    version: 0.4.5

solidity:
  import_remapping:
    -<EMAIL_ADDRESS>    -<EMAIL_ADDRESS>
default_ecosystem: ethereum

ethereum:
  default_network: mainnet-fork
  mainnet:
    default_provider: infura
  mainnet_fork:
    default_provider: foundry

arbitrum:
  default_network: mainnet-fork
  mainnet:
    default_provider: infura
  mainnet_fork:
    default_provider: foundry

compile:
  exclude:
   - "*.json"

What went wrong?
INFO (ape-test): Submitted https://sepolia.etherscan.io/tx/0x4d5d7ecb75ee5a93139b73a675e6c4ee51432682fa66a657997b07834496783c
Confirmations (2/2): 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:28<00:00, 14.30s/it]
INFO (ape-test): Confirmed 0x4d5d7ecb75ee5a93139b73a675e6c4ee51432682fa66a657997b07834496783c (total fees paid =<PHONE_NUMBER>6220872)
SUCCESS (ape-test): Contract 'ERC20' deployed to: 0xE288507Fe67f7e935C4c28790Cc1f943Ab8DB6CE

  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape/_cli.py", line 37, in invoke
    return super().invoke(ctx)
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape_run/_cli.py", line 74, in invoke
    return super().invoke(ctx)
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape/cli/commands.py", line 127, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/m/dev/points/ape-test/scripts/deploy.py", line 42, in deploy_token
    token = account.deploy(
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape/api/accounts.py", line 230, in deploy
    txn = contract(*args, **kwargs)
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape/contracts/base.py", line 833, in __call__
    return super().__call__(*args, **kwargs)
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape/api/address.py", line 105, in __call__
    return self.provider.send_transaction(txn)
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape_ethereum/provider.py", line 864, in send_transaction
    txn_hash = self.web3.eth.send_raw_transaction(txn.serialize_transaction())
  File "/home/m/dev/points/ape-test/.venv/lib/python3.10/site-packages/ape_ethereum/transactions.py", line 60, in serialize_transaction
    raise SignatureError("The transaction is not signed.")

ERROR (ape-test): (SignatureError) The transaction is not signed.

Please include information like:

what command you ran

    token = account.deploy(
        project.ERC20.deploy(_NAME, _SYMBOL, _INITIAL_SUPPLY, _NAME_EIP712, _VERSION_EIP712, sender=account)
    )

How can it be fixed?
No idea

ahhh, I think I see what happened
if you break apart your command a bit, this is what you did:
    # deployed once (this was successful btw)
    token = project.ERC20.deploy(_NAME, _SYMBOL, _INITIAL_SUPPLY, _NAME_EIP712, _VERSION_EIP712, sender=account)
    # you tried to deploy again, but this time using `token` contract instance as the input arg
    ... = account.deploy(token)

Why AccountAPI.deploy accepts a ContractInstance as an argument is beyond me, it should probably raise a different error, but for now I woukd just not have the second deploy

code snippet is from a yearn repo, but I will retry with the one you provided

I guess the original snippet creates two tx, one singed, and the other not.

yes, that's exactly what's happening, but gonna keep this one open for now because that error message should not be happening


Add Line Bookmark Construct and Move to Next/Previous Bookmark Action

Add a construct of a "bookmark", i.e. a way to mark a line in the code editing window, and the ability to jump to previous or next bookmarked lines.
I suggest the shortcut of "ctl-b" to bookmark a line, and "ctl-n" to jump to the next bookmarked line, and "ctl-p" to jump to the previous bookmarked line.
Each tab should have it's own bookmark namespace.
I see the "breadcrumb" concept in the Visual Studio Code, but I think it's a different concept.
https://code.visualstudio.com/updates/v1_26#_breadcrumbs
What I'm asking for is a way to mark locations in a long file, and to jump between next and previous locations.  When I'm working on a big file, I need a way to easily jump between important sections.  Other editors like SublimeText and Teradata SQL Assistant have this functionality, and it's been very useful in the past.
-Add Bookmark
-Delete/Remove Bookmark
-Jump to Next
-Jump to Previous
If I add or remove lines, the bookmark should stay with the code on the line, not with the absolute line number.  For example, if I put a bookmark on line 500, and then I delete lines 200 to 210, the code marked by the bookmark will then be on line 490. (or, is it line 489?)

Closing as duplicate of https://github.com/microsoft/azuredatastudio/issues/13645


changed pokemon count to include eggs

Short Description: changed pokemon count to include eggs
Changes:
added eggs to count

👍
Working. Thank you!

👍

This is actually part of #249, closing.


GSoC Idea Proposal 2020 -  Implement event loop telemetry

GSoC Idea Proposal 2020 -  Implement event loop telemetry
Hello community I am Shashank J, a final year student pursuing the course of Computer Science and Engineering at Sai Vidya Institute of Technology (VTU) -  Bangalore and a follower of Pieter Hintjens.
I have strong knowledge on Java and good knowledge about AMQP, Asynchronous messaging / networking libraries (ZeroMQ), Zero-Copy, Message Queuing , different types of messaging patterns like PUB-SUB, PUSH-PULL, REQ-RES, PAIR, Polling, Majordomo, Suicidal Snail,B-STAR, TItanic patterns etc. I am also familiar with JMH, JMeter and JProfiler. I started using netty few months back and have built some IoT releated projects (Fog Computing) which requires communication with various level of devices. I would be really interested to work with the community and provide long term support to this project.
Here s my proposal for GSoC 2020 : Link to Proposal - Implement event loop telemetry
I am thinking to use messaging (internal and external) mechanisms  to achieve the metrics counter effect.
More details are discussed in the provided google doc.
I would really love your feedback on this proposal and any suggestions or add-ons for this proposal are always open.

JVM Communication is a nice way but much data can it extract from Netty?

JVM Communication is a nice way but much data can it extract from Netty?

Yes, with JVM IPCs using a memory mapped file we can retrieve about 10~14 million messages per second and with average latency in less than 100 nano seconds.
I am planning to benchmark on raw bytes as most of the time is spent in serializing and deserilizing the message. I believe benchmarking on the messages after serializing  / deserilizing could provide extra overhead which will give us an average latency in few milliseconds.

On different methods like RMI, TCP/IP communication without any persistence I found that inter process communication (JVMs) with a memory mapped file performs significantly faster and is more memory efficient. @hyperxpro What is your idea on this ?

@shashank-shark Aeron Is currently using the IPC strategy (https://github.com/real-logic/aeron/wiki/Monitoring-and-Debugging#counters) to expose counters, but requires Unsafe in order to perform atomic operations over the underline bytes of MappesByteBuffer.

JVM Communication is a nice way but much data can it extract from Netty?

Yes, with JVM IPCs using a memory mapped file we can retrieve about 10~14 million messages per second and with average latency in less than 100 nano seconds.
I am planning to benchmark on raw bytes/messages first and then move to benchmark the messages after their serialization / deserilization and this approach would also give us the average serialization/deserilization latency as well.

Did you benchmark these numbers?

@hyperxpro i won't focus on the perf of such mechanisms (others has already demonstrated the effectiveness of them, see my previous comment about it) but on which kind of benefits and cost it brings if compared to other mechanisms.

@franz1981 yes the points that you've listed are crucial when dealing with IPCs and multithreaded environments. Using Unsafe is the best way to modify an object in memory with less cost.
Adding to the first point, here I would try to use a different atomic counter for different events and let the thread allocated for this event in a handler update the counter atomically(1) using Unsafe and the counters offset address. With this I think I would achieve event level versioning.
For thread level versioning I am currently working on the data structure part which maps threadId to an atomic incrementer as threadId would remain same and unchanged during the life cycle of the thread. But still once the thread is terminated, the threadId may be reused by some other newly created thread. Any suggestions ?
Adding to the second point, I am thinking of implementing a shared queue and a broker which provides persistence (of 1 second) to the messages that are sent by the producer running on target netty application. The remote JVM (not part of netty application) which is connected to another end of this queue gets these updates every 1 second and to check the liveliness of the process or to detect a crash I am thinking to implement heartbeating between the broker and the process. Any suggestions ?

@shashank-shark The problem I see with this approach is that in order to work (without considering the new VarHandle methods on next jdk that will introduce it) is that it needs Unsafe to work fine: it means Android and IBM platfoem won't work (maybe :P). Netty features IMHO should aim to be available for most possible platforms (see how PlatformDependent works) and hence an approach that work just with Unsafe needs to be careful evaluated: unless you are able to make it work across different JVM impls without relying on it I am not sure the IPC approach is feasible...but I am happy to be wrong :)

Yes using Unsafe is not much fruitful as Unsafe is already deprecated in JDK 11 and is available in jdk.unsupported module and even I doubt that Unsafe won't be supported in future by many JDK vendors out there.
The solution I was able to think of was to determine the JDK version at runtime and then move on to implement the counters. Here are the versions with the plan of implementation.

JDK Version
Plan

JDK 7
To use JNI for counters implementation.

JDK 8
To see if Unsafe is supported by the vendor, if yes then use Unsafe else use JNI.

JDK 9
To use Variable Handlers (VarHandle) to implement counters.

JDK 10 and above
To use Variable Handlers (VarHandle) to implement counters.

Let me try out using IPCs, if this solution is not feasible then I am planning to use message queues with shared memory (which I described in my previous comment) and let you know.
By the way, can I confirm that use of JNI is allowed to achieve this target and are there any additions that I can make to the above table ?

JVM Communication is a nice way but much data can it extract from Netty?

Yes, with JVM IPCs using a memory mapped file we can retrieve about 10~14 million messages per second and with average latency in less than 100 nano seconds.
I am planning to benchmark on raw bytes/messages first and then move to benchmark the messages after their serialization / deserilization and this approach would also give us the average serialization/deserilization latency as well.

Did you benchmark these numbers?

Yes, these numbers could be number of :

Threads.
Channels.
Events.
Iterations (depending on use case).

Let's close this... the students were selected for GSOC


[Bug] Request path /schedule/0

Our servers are getting a lot of requests for /schedule/0 :') In fact, it's the highest number of invocations out of any other routes. Let me know if this is intended please.

That's definitely not intended. Let me kill the bot for a second on my side and see if that stops it. Though it's possible it's someone else's deployment.

Hey @daenney, we're still getting a consistent flow of requests at 10 second intervals and I'm not entirely sure it's someone else's deployment since the user agent cites it as this repository. Can you investigate a bit more?

The particular user-agent of the bot was changed from gdq to gdqbot in 2022 (https://github.com/daenney/gdqbot/commit/c87260ceeac6647b07fdbeb8578e4e280f035c85). I forgot to make the user-agent configurable and mandatory to be set, so unfortunately it always shows my repo :facepalm:.
I've double-checked my box, it's not running there. For the record, the possible IPs (both on Hetzner) on my side are:

<IP_ADDRESS>
2a01:4f9:3a:1e48::

Do you see the traffic coming from there?
This library used to, once upon a time, hit and scrape /schedule until I discovered the tracker API. That was changed in https://github.com/daenney/gdq/commit/29d1e3dbdc4af8167d5b468f34c1717503c84a01, in 2021, and eventually released as v2. I have however no idea why it would be hitting /en/schedule, that's not something I ever had.
For this request to still be showing up, someone has to be running an ancient deployment of the bot, before I upgraded it to using the v2 version.
What I can immediately do is make the user-agent configurable and without a default, so any additional operator will need to explicitly set their own. It won't fix the current situation, but you should then at least be able to block the old one, gdqbot (+https://github.com/daenney/gdq).


Add NEXT button for annotator and validator

Front-end

[x] Add NEXT with save and NEXT without save functionality. It's ok to add dropdown on FINISH LABELING button

** Back-end**

[ ] Add endpoint to annotation service to get NEXT task and PREVIOUS task. This endpoint receives id of current task and must return object: {next_task: ..., previous_task: ...}

Algorithm how to select next and previous tasks
Let's simplify and get ALL tasks by user_id ordered by same field (id?) and get 2 nearest to current.

@khyurri  Do we need to control the statuses of returned tasks?

back-end side is ready: https://github.com/epam/badgerdoc/pull/371

Verified


Long list item sequences in table cells break page layout

Hello,
Technical documents usually contain  some kind of changelog where every modification of the document is summarized for each released issue.
A sample changelog table is:

Issue
Date
Change record
Author

0.1
19/04/2020
Initial release
Alice

1.0
20/04/2020
"..."
Bob

In the change record column, I typically list every comment identifier I took into account with a short description. So, this column often contains a list.
Across major revisions of technical documents, this list can become huge. I have noticed asciidoctor-pdf struggles to layout such changelog table: the cell content flows out from its cell boundaries to the bottom of the page and overlaps with the content of the next pages. See below image for an example

The source for this example
sample.txt
I tried manually adding page breaks into the offending cells, with no success.
I use asciidoctor/docker-asciidoctor to generate my documents.

bash-5.0$ asciidoctor-pdf --version
Asciidoctor PDF 1.5.0 using Asciidoctor 2.0.10 [https://asciidoctor.org]
Runtime Environment (ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux-musl]) (lc:UTF-8 fs:UTF-8 in:UTF-8 ex:UTF-8)

This is a duplicate of #1623, which was resolved in master. I would consider back-porting it to the 1.5.x line.
That fix merely truncates the cell. A cell cannot be taller than a single page. That's because the PDF generator we use doesn't support splitting a cell across pages. This limitation is mentioned in the README. You'll have the split the cell manually my moving some of the content to another row until the cell fits on a single page.

Ok, thanks for the workaround.
I am using asciidoctor-docker and I am not tied to a particular version. I can test the fix as soon as an updated docker image is available.


Understanding the POST version of the /login route

Understanding the POST version of the login route/page was more difficult than the GET version because it required me to deal with sessions. I still have a limited and vague idea of how sessions work, but I would like to understand it better given more time. I was able to receive the form data from the POST method and compare that against hard-coded values for the username and password. I would like to work on this further and authenticate through a database and perhaps encrypt the data. I also have no idea how to do this, but I do know it is somehow related to HTTPS. Given more time I would like to add these features.

Creating new issue for login security/authentication


♻ TinaCMS - Customise the reference field selector

CC @wicksipedia @tkapa @Ben0189
Hey Team,
Description:
Following https://github.com/tinacms/tinacms/pull/4701 we would now like to be able to customise the selector
Tasks

[ ] Should be able to style the option to include more data than just the filename

[ ] Extend the reference field type and add the following or similar
[ ] an options object that has the following or similar

[ ] label: ({field}) => string - The label to be displayed
[ ] search:string[] - The field's that should be used to index/search on
[ ] optionOverride - A custom component that will replace the default option that is rendered. Maybe experiment with replacing the whole List component?

Screenshots

Figure: MacOS

Figure: Windows

Progress update

[ ] Add display option property to schema tools for reference field
[ ] Validate new display option property
[ ] Refactor graphql query to retrieve display option property
[ ] Refactor reference field selector component to display content based on option property

Before:

After:

@Ben0189
Please note that a Done video is required.

[ ] Done video

Done video for new reference field - https://www.youtube.com/watch?v=WjxYTxlnZMU

Documentation pr still under review - https://github.com/tinacms/tina.io/pull/2041

Documentation pr still under review - tinacms/tina.io#2041

PR has been merged into main


ibmcloud: add e2e test ibmcloud provisioner

Fixes: #490
Signed-off-by: Qi Feng Huo<EMAIL_ADDRESS>
[ ] Provisioner https://github.com/confidential-containers/cloud-api-adaptor/issues/490

[x] VPC & Subnet
[x] CoS and PodVM
[x] IKS Cluster
[ ] Operator
[x] Use prop files rather than system variables
[x] Expose the ability to developer
[ ] Test

Self managed cluster will be covered by story #620

@wainersm I'm going to rebase the code and pick up your changes for properties file to replace system ENV. Although I have not tested completely, I think its ready for review. May you please help check it?

I'm going to test it after rebase and pick https://github.com/confidential-containers/cloud-api-adaptor/pull/650, but think it's time for initial review @liudalibj @stevenhorsman

Hey @huoqifeng - just to check that I understand the conceptual approach this is shifting to:

Move the cluster provisioning logic for all platforms into the provutils to decouple it from the testing code
We can then use this isolation to provision peer pod enabled clusters with the new cluster-provisioner binary
Providing I've understood this then I definitely like this approach and the decoupling and can see that it would be very useful to me with both an end user and developer who runs the test hats on!

From the code point of view I guess my initial concern is how large the ibmcloud_provisioner.go is an will become once we add self-managed support in, so an approach like @jtumber-ibm suggested in https://github.com/confidential-containers/cloud-api-adaptor/issues/640 might help towards that.
It will also be great to see how this can align with the properties work that Wainer did and is planning in the kustomize area.

Hey @huoqifeng - just to check that I understand the conceptual approach this is shifting to:
* Move the cluster provisioning logic for all platforms into the `provutils` to decouple it from the testing code

* We can then use this isolation to provision peer pod enabled clusters with the new `cluster-provisioner` binary
  Providing I've understood this then I definitely like this approach and the decoupling and can see that it would be very useful to me with both an end user and developer who runs the test hats on!

From the code point of view I guess my initial concern is how large the ibmcloud_provisioner.go is an will become once we add self-managed support in, so an approach like @jtumber-ibm suggested in #640 might help towards that.
It will also be great to see how this can align with the properties work that Wainer did and is planning in the kustomize area.

@stevenhorsman yes, the intention is exactly what you mentioned, I like @jtumber-ibm code structure, I actually used file ibmcloud_provisioner_iks.go and ibmcloud_provisioner_selfmanaged.go to separate IKS and self-managed cluster originally but worried about confusion other providers developer :-) . I was not using provisioner as package name but used provutil just intend to avoid name conflict in main_test.go. I'm glad to rename the package name provutil to provisioner and bring back the separated clusters file. And yes, I'll rebase code to pick @wainersm change first and then do the refactoring.

I'll use several small PR to prepare for the change, https://github.com/confidential-containers/cloud-api-adaptor/pull/657 is to refactor e2e test code firstly.

I guess some changed happened in CI recently? checking why errors happened like below in several job:
cat: .git-commit: No such file or directory

I guess some changes happened in CI recently? checking why errors happened like below in several job:
cat: .git-commit: No such file or directory

make .git-commit ?

I guess some changes happened in CI recently? checking why errors happened like below in several job:
cat: .git-commit: No such file or directory

Yeah, maybe the image target needs a dependency on the .git-commit target now?

Hey @huoqifeng - I've had a go at running this PR and I think there are some issues that have crept in before your PR around the single image build and stuff that we need to resolve first.
e.g. as we saw earlier the image target should have a depency on the .commit.tmp which I'll raise separately tomorrow.
but the main problem is that the CLOUD_PROVIDER is largely ignored and all the cloud-provider tags passed through in GOFLAGS.
Just to be clear - these aren't issues with your PR, but I think we need to resolve them before we can test it properly, so sorry for the delay this might cause.

Tested as below:
export DEBUG="true"
export CLOUD_PROVIDER="ibmcloud"
export TEST_E2E_PROVISION_FILE="/root/provision_ibmcloud.properties"
export TEST_E2E_PODVM_IMAGE="/root/e2e-test-image-amd64-20230308-1.qcow2"
export TEST_E2E_PROVISION="yes"

# ./cluster-provisioner -action=provision
&{ApiKey:xxxxxxxx Bucket:peerpod-cos-bucket ClusterName:e2e-test2 CosApiKey:xxxxxxxx CosInstanceID:crn:v1:bluemix:public:cloud-object-storage:global:a/cdd0a83a81fd38ed101096a049942e6d:1bd99fa3-2a6b-45f7-815d-2271033fdaa5:: CosServiceURL:s3.jp-tok.cloud-object-storage.appdomain.cloud SecurityGroupID: IamServiceURL:https://iam.cloud.ibm.com/identity/token IksVersion:1.26 InstanceProfile:bz2-2x8 PodvmImageID: PodvmImageArch:s390x Region:jp-tok ResourceGroupID:33e11ea6cbf340f2bfb6c8414f511b8d SshKeyID:r022-3ec05389-9662-44a9-8f84-71edde2e6af3 SubnetName:e2e-test2-vpc-subnet SubnetID: VpcName:e2e-test2-vpc VpcID: VpcServiceURL:https://jp-tok.iaas.cloud.ibm.com/v1 WorkerFlavor:bx2.2x8 Zone:jp-tok-1 WorkerCount:1 IsSelfManaged:false IsProvNewVPC:true IsProvNewSubnet:true IsDebug:true VPC:<nil> ClusterAPI:<nil>}
initClustersAPI()
initVpcV1()
CreateVPC()
Creating VPC e2e-test2-vpc in ResourceGroupID 33e11ea6cbf340f2bfb6c8414f511b8d.

Created VPC with ID r022-d41cf41b-f534-4e78-bff1-e3f6f092ef64 in ResourceGroupID 33e11ea6cbf340f2bfb6c8414f511b8d.

Got VPC default SecurityGroupID r022-8a03cb89-7e15-4bea-a18e-8ec7543ba307.

waiting vpc ready before create subnet
Creating subnet e2e-test2-vpc-subnet in VPC r022-d41cf41b-f534-4e78-bff1-e3f6f092ef64 in Zone jp-tok-1.

Created subnet with ID 02e7-e6b86f45-61fc-4b91-b090-554641860513.

CreateCluster()

# ./cluster-provisioner -action=provision
&{ApiKey:xxxxxxx Bucket:peerpod-cos-bucket ClusterName:e2e-test2 CosApiKey:xxxxxxx CosInstanceID:crn:v1:bluemix:public:cloud-object-storage:global:a/cdd0a83a81fd38ed101096a049942e6d:1bd99fa3-2a6b-45f7-815d-2271033fdaa5:: CosServiceURL:s3.jp-tok.cloud-object-storage.appdomain.cloud SecurityGroupID: IamServiceURL:https://iam.cloud.ibm.com/identity/token IksVersion:1.26 InstanceProfile:bz2-2x8 PodvmImageID: PodvmImageArch:s390x Region:jp-tok ResourceGroupID:33e11ea6cbf340f2bfb6c8414f511b8d SshKeyID:r022-3ec05389-9662-44a9-8f84-71edde2e6af3 SubnetName:e2e-test2-vpc-subnet SubnetID: VpcName:e2e-test2-vpc VpcID: VpcServiceURL:https://jp-tok.iaas.cloud.ibm.com/v1 WorkerFlavor:bx2.2x8 Zone:jp-tok-1 WorkerCount:1 IsSelfManaged:false IsProvNewVPC:true IsProvNewSubnet:true IsDebug:true VPC:<nil> ClusterAPI:<nil>}
initClustersAPI()
initVpcV1()
CreateVPC()
Creating VPC e2e-test2-vpc in ResourceGroupID 33e11ea6cbf340f2bfb6c8414f511b8d.

Created VPC with ID r022-d41cf41b-f534-4e78-bff1-e3f6f092ef64 in ResourceGroupID 33e11ea6cbf340f2bfb6c8414f511b8d.

Got VPC default SecurityGroupID r022-8a03cb89-7e15-4bea-a18e-8ec7543ba307.

waiting vpc ready before create subnet
Creating subnet e2e-test2-vpc-subnet in VPC r022-d41cf41b-f534-4e78-bff1-e3f6f092ef64 in Zone jp-tok-1.

Created subnet with ID 02e7-e6b86f45-61fc-4b91-b090-554641860513.

CreateCluster()
Creating cluster e2e-test2.

Will the cluster-provisioner binary also use libvirt?
If so, could you update this line
https://github.com/confidential-containers/cloud-api-adaptor/blob/cbcc961d6721c77a1c9e0c446ddfdcc3002e2076/Makefile#L64-L66
Like this?
cloud-api-adaptor cluster-provisioner: GOOPTIONS := $(subst CGO_ENABLED=0,CGO_ENABLED=1,$(GOOPTIONS))

Changed, thank you @yoheiueda

@stevenhorsman @wainersm may you please help review the PR? An end to end test will still need merge https://github.com/confidential-containers/cloud-api-adaptor/pull/668 to get operator deployed correctly. This PR will just create VPC/Subnet and clusters.

@stevenhorsman @wainersm may you please help review the PR? An end to end test will still need merge #668 to get operator deployed correctly.

Yes, I hope to get around to trying it out today - sorry, tons of meetings and a big backlog of review requests.

Merge the PR so that others might pick the enhancements issue if I don't have capacity. I don't want to be bottleneck here :-)


Update user's settings.json from vim

Hi, I started using the recent feature of displaying "inlay hints" within the source code and it's great too see all that info real quick, but it would be even better if I could toggle its visibility, since it also makes the code a bit cluttered. Unfortunately there's no toggle command, only a key in settings.json file.
My hope would be to be able to do something like this:
nnoremap [i <Cmd>call VSCodeSetUserSettingsValue('editor.inlayHints.enabled', 1)<CR>
nnoremap ]i <Cmd>call VSCodeSetUserSettingsValue('editor.inlayHints.enabled', 0)<CR>

So, is there such a function available? If there isn't, would someone kindly give me some hints as to how I could implement such function? I'm not really familiar with VSCode's extension APIs.

https://github.com/vscode-neovim/vscode-neovim/issues/852#issuecomment-1106592180

It looks like the extension API supports updating config values via the update() function: code.visualstudio.com/api/references/vscode-api#WorkspaceConfiguration
You could make a very simple extension / modify this extension to add a vscode command that calls that function, and then call that command via VSCodeNotify.
I know "make a new extension" is not very helpful, but to answer your question yes it certainly seems possible.

There is an extension called "Toggle" (Marketplace, Github) which could be the missing piece.
It allows to map keybindings in VS Code to toggle between settings in the users's settings.json. What to toggle and which keybinding is used is defined in the keybindings.json as follows (modified example from the extension's readme):
{
    "key": "F3",
    "command": "toggle",
    "args": {
        "id": "minimap",
        "value": [
            {
                "editor.minimap.enabled": true
            },
            {
                "editor.minimap.enabled": false
            }
        ]
    }
}

Before attempting to map the function in my init.vim I tried to validate it using the VIM Command Line in VS Code. However, if I try to pass arguments like …

call VSCodeNotify('toggle', { 'id': 'minimap' } )  or
call VSCodeNotify('toggle', { 'args': { 'id': 'minimap' } })

… this error message is shown:

Please make sure your 'args' is not empty

Yet, there is no error in VS Code's output.
What would be the right way to call the function?
Another idea would be to use the Toggle extension to register a keybinding (like F3 in the example above) and then have neovim send a simulated keystroke to VS Code.
Is this even possible using VSCodeNotify()?
I could not find anything similar done in the advanced examples.
I'd be very grateful for any input on that. Thanks a lot in advance.

https://github.com/vscode-neovim/vscode-neovim#invoking-vscode-actions-from-neovim

Yes it is possible to use Toggle or related plugins (like settings cycle) that expose a vscode command, to call that command in nvim.

I could not find anything similar done in the advanced examples. I'd be very grateful for any input on that. Thanks a lot in advance.

You should set the args as a lua table:
local args = {
    id="minimap",
    value={
        -- Need ["key"] syntax because dot in identifier is not valid
        {["editor.minimap.enabled"]=true},
        {["editor.minimap.enabled"]=false},
    }
}

For info, I needed to add a keybind that changed a setting value and notify a command. I just used notify first to set my settings to the desired value (with each element in value with the same value), then use the notify a second time.


MethodDef::Params() should return a pair of iterators

All collection methods return a pair of iterators, with the exception of MethodDef::Params(), which returns a const vector.
Fix to make it act like all the others for consistency. Otherwise, consumers have to write a range specialization just for this one case.

There are other places, need to fix those too.


[c-ares] Update to 1.31.0

Update c-ares to 1.31.0.
No feature needs to be tested, the usage test passed on x64-windows (header files found):
The package c-ares provides CMake targets:

    find_package(c-ares CONFIG REQUIRED)
    target_link_libraries(main PRIVATE c-ares::cares)

[x] Changes comply with the maintainer guide.
[x] SHA512s are updated for each updated download.
[ ] ~The "supports" clause reflects platforms that may be fixed by this new version.~
[ ] ~Any fixed CI baseline entries are removed from that file.~
[ ] ~Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and commit~ting the result.
[x] Only one version is added to each modified port's versions file.

Are you still working on this?


test require ref doesn't support variable expansion nor dynamic ref

In a test I have library requires:
 require:
   - url: https://github.com/RedHat-SP-Security/keylime-tests
     ref: ${REF_BRANCH}
     name: /Library/test-helpers

where REF_BRANCH is defined in a plan.
However this doesn't work as I am getting
fail: Reference '${REF_BRANCH}' for library 'keylime-tests/test-helpers' not found.

Also, dynamic ref is most likely not supported too.

The following seems to work for me for beakerlib library.
diff --git a/tmt/libraries/beakerlib.py b/tmt/libraries/beakerlib.py
index 8e3c4ff7..6821c423 100644
--- a/tmt/libraries/beakerlib.py
+++ b/tmt/libraries/beakerlib.py
@@ -255,11 +255,30 @@ class BeakerLib(Library):
                     # Use the default branch if no ref provided
                     if self.ref is None:
                         self.ref = self.default_branch
+                    try:
+                        plan = self.parent.parent.plan
+                        dynamic_ref = tmt.base.resolve_dynamic_ref(
+                            workdir=clone_dir,
+                            ref=self.ref,
+                            plan=plan,
+                            logger=plan._logger
+                            )
+                    except tmt.utils.FileError as error:
+                        raise tmt.utils.DiscoverError(
+                            f"Failed to resolve dynamic ref of '{plan_id.ref}'.") from error
                     # Check out the requested branch
                     try:
                         if self.ref is not None:
-                            self.parent.run(
-                                Command('git', 'checkout', self.ref), cwd=clone_dir)
+                            # we won't change self.ref directly since we want preserve a check
+                            # for not fetching two distinct 'ref's. Simply put, only the same
+                            # dynamic_ref can be used by other tests.
+                            if self.ref != dynamic_ref:
+                                self.parent.debug(f"Dynamic 'ref' resolved as '{dynamic_ref}'.")
+                                self.parent.run(
+                                    Command('git', 'checkout', dynamic_ref), cwd=clone_dir)
+                            else:
+                                self.parent.run(
+                                    Command('git', 'checkout', dynamic_ref), cwd=clone_dir)
                     except tmt.utils.RunError:
                         # Fallback to install during the prepare step if in rpm format
                         if self.format == 'rpm':

Regarding variable expansion, I am not really pushing on it as I do not have a use case for it once dynref works. Also, variable expansion could be possibly problematic for tmt test export since variables are typically populated in test plans and the variable won't be expanded during the export.

My attempt to implement dynamic ref support for library dependencies
https://github.com/teemtee/tmt/pull/2240


Web Portal Development

Development of a web portal that supports user authentication, session management and allows users add or delete taxonomies.

https://github.com/Cellular-Semantics/tdt_web_portal created

[ ] User authentication
[ ] Session management
[ ] Display 'My taxonomies'
[ ] Add taxonomy
[ ] Delete taxonomy


Interface support

I decided to add some interface support to the code.  I came up with a few possible solutions:
http://stackoverflow.com/questions/41788679/domain-driven-design-for-node-js/42418735#42418735
There is another implementation with getOwnPropertyDescriptors.
https://www.webreflection.co.uk/blog/2016/03/23/javascript-interfaces
The ideal would be to use a polyfill with both toString and getOwnPropertyDescriptors. So it would be more or less backward compatible.
I am not sure whether I want to put this feature into the class lib. I'll think about this. I did not want to be too oo by implementing this lib. So no interfaces, method override, type check, etc...

To add interface support to classes we need to wrap the constructor. So by instantiating the class we can throw an abstract class instantiation exception. Wrapping the constructor is not a good idea, because I won't be able to extend already existing classes with the class related methods. So I am pretty sure that this approach won't work. I don't want to wrap methods either and I don't want to have auto type check on the arguments.
I need a different approach.

The upper means that I cannot verify the interface by instantiation. All I can do is verifying the interface by adding it to the class.
var i = Interface.extend({
    a: {x: String}
});
var j = Interface.extend({
    b: {}
});
var c = Class.extend({
    prototype: {
        a: function (x){}
    }
});

c.implements(i);
// ok

c.implements(j);
// error: b() is not implemented

There is cache, so if c implements i, then it will be cached until c is modified by a class method. Modification of c invalidates the cache. We can check next time when asked, or we can set this rule permanent. I prefer the latter one. Which means after c.implements(i) is called, c and its descendants MUST implement i at any time.
var i = Interface.extend({
    a: {x: String}
});
var c = Class.extend({
    prototype: {
        a: function (x){}
    }
});

c.implements(i);
// ok

c.prototype.merge({
    a: function (x, y){}
});
// error: a(x,y) does not fulfill the a(x) interface constraint

Interfaces must be immutable, so we can define them only once and they cannot be modified after that. Interfaces can have a single ancestor interface, but the descendant interfaces cannot override or make constraints on the methods defined by the ancestor interface.
var i = Interface.extend({
    a: {x: String}
});

var j = i.extend({
    b: {y: Number}
});
// ok

var k = i.extend({
    a: {y: Number}
});
// error: a(y) does not fulfill the a(x) interface constraint
// error: cannot redeclare the a(x) interface constraint (maybe this latter one is better)

Type check?
I am certain I won't wrap any method, but it doesn't hurt if we provide type checking solutions based on the interface. That means:
var i = Interface.extend({
    a: {x: String}
});
var c = Class.extend({
    prototype: {
        a: function (x){
            this.validate({a: {x:x}});
            // or
            this.validate({a: [x]});
            // or
            this.validate(this.a, arguments);
            // probably the latter one requires the least effort
        }
    }
});

c.implements({
    i:i
});
// alternative approach to name interfaces locally
// not necessary, it might be better to name them the same way we did by the classes

I think that's all related to interfaces and validation. It can be in a separate project as well, maybe i3. I am not sure yet. I have to override merge and absorb for certain, because I need the verification code to put there.

Another possible approach to declare interfaces without type checking:
var i = new Interface({
    a: function (x){}
});

This can be okay as long as we don't need type checking. Maybe we can put the type checking code into this function.
var i = new Interface({
    a: function (x){
        x.implements(j);
    }
});

What if we don't try to implement type check and validation based on this? I'd like to keep it simple, so type check should be added to the actual method as usual:
Q = Class.extend({
    prototype: {
        m: function (x){
            if (!(x instanceof Y))
                throw new InvalidArgumentException("...");
            ...
        }
    }
});

So using interface does not do type check, it just check whether the actual methods are implemented.
iQ = new Interface({
    prototype: {m: function (x){}}
});
Q = Class.extend({
    prototype: {
        m: function (x){
            if (!(x instanceof Y))
                throw new InvalidArgumentException("...");
            ...
        }
    }
}).implement(iQ);

or probably
iQ = new Interface({
    prototype: {m: function (x){}}
});
Q = Class.extend({
    implement: iQ,
    prototype: {
        m: function (x){
            if (!(x instanceof Y))
                throw new InvalidArgumentException("...");
            ...
        }
    }
})

or maybe
iQ = new Interface({
    prototype: {m: function (x){}}
});
Q = Class.extend(iQ, {
    prototype: {
        m: function (x){
            if (!(x instanceof Y))
                throw new InvalidArgumentException("...");
            ...
        }
    }
})

I like this latter one. The only problem that it is not compatible with instanceof, but who cares. :-)
Afaik. interfaces define only methods, but not static methods. So I should not use prototypes in them.
iQ = new Interface({
    m: function (x){}
});

Ofc. we don't have to stick to the old definition, but I think it is fine this way.
This is all good, but I designed this lib to never override the constructor, so it is compatible with every other existing lib. I don't want to give this feature up just to check whether the instantiated class is fully implementing an interface.
I think that was the cause I decided to support only on the fly checks of instances. e.g.
iQ = new Interface({
    m: function (x){}
});
Q = Class.extend({
    prototype: {
        m: function (x){
            if (!(x instanceof Y))
                throw new InvalidArgumentException("...");
            ...
        }
    }
});
P = Class.extend({
    prototype: {
        a: function (q){
            if (!q.implements(iQ))
                throw new InvalidArgumentException("...");
            ...
        }
    }
});

I think this should be the final format. We can add a cache for classes and instances, so using toString by methods and checking the argument list would be necessary only for a single time.

I think we can add support for this on the fly check. I added it to the task list, but it is not a priority task and it should be added only with a proper caching mechanism, otherwise it would be too slow.

iQ = new Interface({
    m: function (x){}
});
Q = Class.extend({
    prototype: {
        m: function (x){
            if (!(x instanceof Y))
                throw new InvalidArgumentException("...");
            ...
        }
    }
});
P = Class.extend({
    prototype: {
        a: function (q){
            if (!q.implements(iQ))
                throw new InvalidArgumentException("...");
            ...
        }
    }
});

Let's check what happens here:

q.implements(iQ) is called - probably not the best naming, but ok for now
iterates through the methods defined in iQ
check where these methods was defined by q
if it was defined in an ancestor, then check the implementation of the ancestor for every method
if one of the methods are not implemented or is an abstractMethod, then it steps to the previous ancestor (which is closer to the actual instance)
the cache meanwhile logs whether a method is implemented in a class
if it doesn't find a class, which implements every methods, then it checks the intance
if the instance implements the missing methods, then it is fine, otherwise, the result will be false

The cache is an interesting topic here. We can do method <-> interface.method cache and we can do class <-> interface cache. I think we should do both, so if we have an abstract class and the instance implements the last method, then the other methods of the class won't be checked again and again. Method level cache guarantees that. Class level cache makes checking a class somewhat faster, but it is probably not as important as method level cache, because there is no string comparison involved there.
I saw somewhere that the arguments of a method can be listed, but I think that was not ES6, but some kind of transpiled stuff, probably typescript. I'll investigate it.

I saw somewhere that the arguments of a method can be listed, but I think that was not ES6, but some kind of transpiled stuff, probably typescript. I'll investigate it.
If I don't find it, then we can do something like this (not sure about the regex flags and the match output):
Function.prototype.listArguments = function listArguments(){
    if (this.args)
        return this.args;
    var str = this.toString();
    var pattern = /^function\s+\w+\s*\(([^)]*)\)/;
    var args = pattern.match(str)[1].replace(" ","").split(",");
    if (args[0] === "")
        args.length = 0;
    this.args = args;
    return this.args;
};

This is fine, if I lazy load this when it is needed.

Should I allow extension by the arg list or should it be a exact match?
I think it is simpler if it is an exact match and probably more reliable as well.
iQ = new Interface({m: function (x,y){}});
({m: function (x){}}).implements(iQ); //false
({m: function (x,y){}}).implements(iQ); //true
({m: function (x,y,z){}}).implements(iQ); //false
({m: function (x,z,y){}}).implements(iQ); //false

I think the o.implements(i) is a bad solution, because I have to override the Object.prototype. What are the other options?
iQ.implementedby(o) - fine, but cannot check multiple interfaces at once (probably no need for that though most of the times
Interface.implementedby.call(iQ,o) - fine
Interface(iQ,iP).implementedby(o1,o2)
I think all of these are fine and we can do something like
Interface.implementedby(o,iQ) too, because we can check whether something has an interface type
I think it is important that iQ instanceof Interface should be true and we should be able to inherit interfaces with extend in a similar way as classes.

ES6 supports some kind of enumeration: function dumpArguments(...args) { I think it would be better to normalize (remove whitespace) and check for string match instead of trying to parser everything. It is compatible with the new features. I need a HashMap(f, argsString) for that. That is cool if we have HashMap support, otherwise it can't be done without a memory leak.

#105 (WeakMap polyfill needed for cache)

It is possible to support ES6 features in the argument list without parsing it. At least I think so. I need to look for ( and ) chars so I will know when the argument list is ended. The only problem when these chars appear in strings, so I guess I have to look for strings as well. Still better than including a complete js parser. I need to check exactly what is supported in ES6 argument lists. Maybe it is a lot easier than I think.


Figure out a way to keep track of attributes

 Issue by brson
Sunday Jun 23, 2013 at 05:25 GMT
For earlier discussion, see https://github.com/rust-lang/rust/issues/7318
This issue was labelled with: I-wishlist in the Rust repository

There are a lot of attributes in use. They are nearly undocumentedand impossible to catalog automatically.
The wiki is out of date: https://github.com/mozilla/rust/wiki/Doc-attributes

Defined in https://github.com/rust-lang-nursery/reference/blob/master/src/attributes.md.
Closing as done.


Exclude files matching ignore patterns (fixes #71)

Do not run the loader against file paths matching ignore patterns within the StandardJS package config.

Do you think maybe a less magical way to do would be to explicitly use webpack's exclude field when configuring the package? e.g.
const pkg = require('./package.json')
// ...
{
// ...
        {
        test: /\.jsx?$/,
        loader: 'standard-loader',
        exclude: pkg.standard.ignore
      }
}
// ...

One concern I have with this PR and my suggestion is that it doesn't guarantee the same matching algorithm used by the standard package, though I'm not sure a good way around that.

I could be wrong, but I believe the webpack loader exclude utilizes regex and StandardJS' ignore relies on globs. So, I don't think we could merely copy the package.json config into the loader exclude. I will test that theory, though.
If you feel like this is a feature that should not be included at all, that is fine. However, it would've been helpful to receive that feedback in #71. ☹
P.S. – I do not know why the tests fail in the CI. They are passing locally for me. I'll try to get them passing.

@timoxley I looked into it a bit more and the two libraries do, in fact, use completely different methods for matching patterns. StandaredJS uses deglob and webpack utilizes RegEx. So, I see no way to share the configuration by passing it directly.
That said, I'm going to close this PR. It is now out-of-date and my use case doesn't appear to be one that aligns with the project. Thanks for considering and publishing the project.


Auto generate nav based on pages Headers 2

          Maybe there is an Asciidoc macro or something that will generate the TOC based on the file. I know there is this: https://docs.asciidoctor.org/asciidoc/latest/toc/ but it doesn't seem to work for XREF links

Originally posted by @gastaldi in https://github.com/quarkiverse/quarkus-renarde/pull/148#discussion_r1266801249

Isn't this done already?
See, for example, https://docs.quarkiverse.io/quarkus-pact/dev/consumer-index.html. I did a quick code search, and all the content in the left is only based on headers, not on manual config.

Or is the idea to also have that content on the right appear on the left? I think that's just a layout thing, not a generation thing, but I'd need to experiment.
Maybe @ia3andy you could provide a screenshot of the intended effect? I looked at https://github.com/quarkiverse/quarkus-renarde/pull/148#discussion_r1266801249 but it wasn't clear enough to me where the nav was, and what was different from the current state.

@holly-cummins so as a User point of view, it seems to me that:

Left nav is top level main menu/nav (currently manual)
Right nav is page deep summary/nav (automatic)

In the left Nav, I would automatically link to all pages and reference h2 (main titles)
In the right Nav, keep it as it is.

Do you think there's value in having stuff over in the right that duplicates stuff on the left? It feels like ... a lot of duplication.
I know it's how Antora pages are usually structured, but that doesn't make it good. :) I wonder if just having everything on the left would be more natural to people who weren't antora-users.
On the left nav, automatically discovering the pages may need some thought, to make sure we

don't break existing navs
do the right thing

We can apply a convention that index.adoc is the top level, but if there's multiple other adocs, we may need to do something clever to order them. I guess requiring the adoc pages to be numbered would sort it out.
I also don't know if antora lets us drop the nav element in antora.yml, but I can do some experiments.
I agree that when viewing an extension, the left bar is pretty sparse, and I definitely agree that filling it in manually down to H2 is (a) boring and (b) a recipe for errors.

I think it looks nice: https://docs.quarkiverse.io/quarkus-renarde/dev/

I agree that the richer navigation on the left is much better than the default. I'm not sure what value the navigation on the right is bringing. It means that to navigate, a user has to flip their attention back and forth between the left and right bars, and a lot of the screen real estate is taken up with repeated content.
I'd suggest, rather than this:

... something like this:

We could add some vertical bar or other divider, and perhaps even some sort of toggle or slider for how many levels to show.

I've just done a quick unscientific survey to see how common the right-bar pattern is. It's the Antora default, which does make it reasonably common.
For example, AWS have it, but they've relabelled the header from 'Content' (which is a totally information-free header) to 'On this page' (which at least gives some indication why there are two nav bars)

Apple do not have two navbars in their developer documentation :)

Microsoft have one navbar and use the right-hand space for extra resources, so that they stay pinned to the top of the page:

Our normal guides have nav on the right, but they do not have nav on the left. That's another option we could do:

@gastaldi @unfromage
The left doc is folded and is acting as shortcut to the main section of the doc, it also help split the content rather than having one huge monolyth.
The right doc is a summary of the content and follow the cursor while scrolling to help navigate it.

I agree that the richer navigation on the left is much better than the default. I'm not sure what value the navigation on the right is bringing. It means that to navigate, a user has to flip their attention back and forth between the left and right bars, and a lot of the screen real estate is taken up with repeated content.

Ah, totally agree :)

The left nav is folded and is acting as shortcut to the main sections of the doc, it also helps split the content rather than having one huge monolith.

The right nav is a summary of the content and follow the cursor while scrolling to help navigate it.

Spliting is fine. But in my experience the left is not folded, and both follow you. I don't find it helpful to split navigation into two parts arbitrarily.


Add Options

Adds a new Regex.Options option set type.
Adds a new, optional options: parameter to Regex.init().
Adds support for the Ignore, IgnoreMetacharacters and AnchorsMatchLines options.

Fixes #3.
To do:

[x] Update the README

I'm pretty happy with how this turned out, so I'm going to go ahed and :boat:
Planning to tag 0.2.5 soon, and if there's any issues with naming we can sort it out before 1.0.


Fix Registration.run()

#87 broke the run() method.  h/t to @ambrosejcarr for finding this.

Codecov Report

Merging #92 into master will not change coverage.
The diff coverage is 0%.

@@          Coverage Diff           @@
##           master     #92   +/-   ##
======================================
  Coverage    68.9%   68.9%
======================================
  Files          24      24
  Lines         981     981
======================================
  Hits          676     676
  Misses        305     305

Impacted Files
Coverage Δ

starfish/pipeline/pipelinecomponent.py
80% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6518d9a...38e3a43. Read the comment docs.


Merkury wifi bulbs

Using a dedicated (to tuya-convert) SD Card and a RPI3b I have successfully flashed eight Gosund plugs.
All moving parts worked in other words.
When attempting to flash the Merkury 9W white wifi bulb, the process just hangs at dots going across the screen while waiting for the device to connect / appear.
I have confirmed that the Tuya Smart app can connect and control the bulbs.
I will continue trying some of the notes / misc ways people are reporting success after work today.

kueblc -  will the newer process (smart-link-py) maybe have different results?

It may, smart-link-py uses two strategies (broadcast and multicast) to pair devices as opposed to just broadcast. It may also resolve some timing issues, certainly worth a try.

Same steps as original project? @kueblc

Yes

@kueblc - I do have the Vtrust recovery AP and I can get to the device info page.  I understand this is a good finding. However, I couldn't get the curl command to complete the flash, nor would (re)running the ./start_flash make any difference.  In both cases, I attempted flash3 without certainty it is correct.  From the info page, I read that available storage was 512K, not 1MB. Does this impact which method? Should my next step be flash method other than 3, or possible the smart-link-py branch?

No progress with the smart-link-py on the previous attempted bulb, or a fresh one.  The fresh one has this info available...... Wifi firmware version 1.1.0.  MCU module firmware version 1.1.0
Observation.... It seems these two bulbs are picking up an IP from my DHCP server before they pickup on the vtrust-flash AP.  I disconnected the DHCP while attempting, no difference, but see next note.
Also, the fresh bulb stops blinking as soon as it pairs with the tuyasmart app.  Previously I thought, on my first attempt, when it stopped blinking it was out of pairing mode all together.  On the first attempt, even though it stopped blinking (as if it was paired) it didn't register as paired with the flashing process.  I'm going to take the fresh bulb offline and attempt to really scrub the process.  Stay tuned.

Yes, if it pairs it will stop blinking. Feel free to share your logs for additional feedback.
Check if your wifi log shows the device's mac address connecting. This would indicate the pairing was successful.
Check also to see if the web log shows any activity. If it does, please share. If it doesn't, see if you can capture what's happening on the network with tcpdump or Wireshark on the device running the access point.

Info page from device:
VTRUST-FLASH 1.0
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://<IP_ADDRESS>/get?read=HEX-ADDRESS
ChipID: 7eb841
MAC: 5C:CF:7F:7E:B8:41
SdkVersion: 1.5.3(aec24ac9)
BootVersion: 4
BootMode: 1
FlashMode: 1M QIO @ 40MHz
FlashChipId: 1440c8
FlashChipRealSize: 1048576
FlashChipSize: 1048576
FlashChipMode: 0
system_upgrade_userbin_check: user2 0x81000

This is great, you've successfully loaded the flash stub. You can now backup your firmware and replace it with your choice of alternatives. Please let us know what trouble you experienced and how you resolved it for future reference.

wlan0: STA 5c:cf:7f:7e:b8:41 IEEE 802.11: associated
wlan0: AP-STA-CONNECTED 5c:cf:7f:7e:b8:41
wlan0: STA 5c:cf:7f:7e:b8:41 RADIUS: starting accounting session 0DB96D51CEF1F8A7
wlan0: STA 5c:cf:7f:7e:b8:41 WPA: pairwise key handshake completed (RSN)
wlan0: interface state ENABLED->DISABLED
wlan0: AP-STA-DISCONNECTED 5c:cf:7f:7e:b8:41
wlan0: AP-DISABLED
nl80211: deinit ifname=wlan0 disabled_11b_rates=0

Traceback (most recent call last):
File "./fake-registration-server.py", line 132, in
main()
File "./fake-registration-server.py", line 128, in main
tornado.ioloop.IOLoop.current().start()
File "/usr/local/lib/python3.5/dist-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 421, in run_forever
self._run_once()
File "/usr/lib/python3.5/asyncio/base_events.py", line 1388, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.5/selectors.py", line 445, in select
fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt

I haven't been able to do anything even though I have the recovery AP.  The logs above are from me attaching my computer to the recovery ap, and attempting the curl command to flash.  It says it will flash and restart, but when it restarts, it is still in recovery mode, and not flashed.

wlan0: STA 5c:cf:7f:7e:b8:41 IEEE 802.11: associated

Is 5c:cf:7f:7e:b8:41 the mac address of the device you're trying to flash?

Traceback (most recent call last):
File "./fake-registration-server.py", line 132, in
main()
File "./fake-registration-server.py", line 128, in main
tornado.ioloop.IOLoop.current().start()
File "/usr/local/lib/python3.5/dist-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 421, in run_forever
self._run_once()
File "/usr/lib/python3.5/asyncio/base_events.py", line 1388, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.5/selectors.py", line 445, in select
fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt

Is this everything that was in the web log?

I haven't been able to do anything even though I have the recovery AP. The logs above are from me attaching my computer to the recovery ap, and attempting the curl command to flash. It says it will flash and restart, but when it restarts, it is still in recovery mode, and not flashed.

Are you still running fake-registration-server.py when you curl? Can you see any requests on your end?

Yes.
There is more, but it is the same (duplicated).
Ummmmm.... no.  I have been running ./start_flash again, should I be running ./fake-registration-server instead?

start_flash should launch fake-registration-server as one of its subprocesses, you should be fine. Check the web log to see the output. The KeyboardInterrupt events are from hitting Ctrl-C to terminate the process. We're interested to see if the device makes a request for thirdparty.bin.

I cleared all logs so I knew what was from today's attempts.  I attempted to run ./fake-registration-server to no avail.  OK.  No new logs to delete.  I ran ./start-flash.  I did not connect another device first.  ./start-flash recognized the bulb online and appeared to be attempting to do something.  Previous attempts informed me it wouldn't be any different.  As I walked out of the room I accidentally hit the light switch.  $@#@^^%#@@......  Turned it back on and BAM..... I see the backup lines flying by.  About 15-20 seconds later the message to flash and have fun.  I flashed it. It rebooted and is now online with tasmota running.

@kueblc - Colin. Before I attempt the second bulb from the pack, what can I do to help?  If it is real complex I need detailed instructions.  Or have I provided enough for you to pin point what's going on? If need be, I can hold off on the second bulb so I can test any update you may make?  Just let me know.

You'd need to be running the AP vtrust-flash before running fake-registration-server. The start_flash script takes care of this as well as creating logs from the output of the subprocesses, which is why you didn't get a log when you ran fake-registration-server by itself.
So just try running start_flash again for the second bulb and proceed as normal. Sounds like you've got the hang of it if you had flashed 8 Gosund plugs successfully. I can't say why this one device was more problematic, could be just a fluke.

Yes.  I do have a grasp. However, others have reported very similar struggles.  I will proceed with second bulb and see how it goes.

Yes. I do have a grasp. However, others have reported very similar struggles. I will proceed with second bulb and see how it goes.

The process is rather amazing. Some devices start faster than others. I have stopped the flashing process about 8 times during the process of reading, or paused, and I have easily recovered from each and every time.  Even when I rebooted the device (mid process) and had no lights and responses- it worked perfectly (and instantly as the device was already waiting for firmware).

Second bulb flashed successfully.....
Steps:

Place device in pairing mode.
run ./start_flash (no other devices connected)
Pairing was FAST.... < 10 seconds.

so, now using this version of TUYA-CONVERT https://github.com/kueblc/tuya-convertit's possibile to flash via OTA also when installed TUYA firmware is v.1.0.5?

I have since learned each device's FIRMWARE revision is different. The only way to know is to try.