Automatically discover a11y text

In discussion with @jessegreenberg, we saw that visual textual information was not exposed to the a11y tree by default.  We had an idea to pluck button/checkbox labels and move them to the PDOM automatically as a default.

It seems like a small amount of effort could dramatically improve our screen reader coverage.  But this issue seems like it is complex enough that it should be moved to an iteration + subteam.  So I'll propose it on the upcoming priorities board, and unassign myself.

This is a great idea @samreid and could lead to knowledge sharing around creating names for objects that work well in both the visual space and the described space. Totally, agree that default on-screen text is better than no text and step in the right direction.

@terracoda indicated difficulty in designing the keyboard traversal order and separating play area vs control area (see https://github.com/phetsims/projectile-data-lab/issues/107) in Projectile Data Lab because the a11y text is all blank:

Here is a patch that takes information from the PhET-iO IDs to populate the a11y view:

Subject: [PATCH] Pass launchButtonEnabledProperty directly to the launchButton.enabledProperty, see https://github.com/phetsims/projectile-data-lab/issues/141
---
Index: js/accessibility/pdom/ParallelDOM.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/accessibility/pdom/ParallelDOM.ts b/js/accessibility/pdom/ParallelDOM.ts
--- a/js/accessibility/pdom/ParallelDOM.ts	(revision 42fefc31fc2910edfd75a72c69e33e78b088c84a)
+++ b/js/accessibility/pdom/ParallelDOM.ts	(date<PHONE_NUMBER>191)
@@ -141,6 +141,7 @@
 import TProperty from '../../../../axon/js/TProperty.js';
 import isSettingPhetioStateProperty from '../../../../tandem/js/isSettingPhetioStateProperty.js';
 import Bounds2 from '../../../../dot/js/Bounds2.js';
+import { animationFrameTimer } from '../../../../axon/js/imports.js';

 const INPUT_TAG = PDOMUtils.TAGS.INPUT;
 const P_TAG = PDOMUtils.TAGS.P;
@@ -632,6 +633,16 @@
     this.focusHighlightChangedEmitter = new TinyEmitter();
     this.pdomDisplaysEmitter = new TinyEmitter();
     this.pdomBoundInputEnabledListener = this.pdomInputEnabledListener.bind( this );
+
+    animationFrameTimer.setTimeout( () => {
+      if ( this.accessibleName === null && this.tandem && this.tandem.supplied ) {
+        let text = this.phetioID.substring( this.phetioID.lastIndexOf( '.' ) + 1 );
+        if ( text.endsWith( 'RadioButton' ) ) {
+          text = text.substring( 0, text.length - 'RadioButton'.length );
+        }
+        this.accessibleName = text;
+      }
+    }, 1000 );
   }

   /***********************************************************************************************************/

With this patch, the a11y view looks like this:

Would be good to get thoughts from @taliesin about whether this is helpful enough or from @jessegreenberg about how we might bring an idea like this to production (even if only for team/development purposes).

The proposal above is not committed, it is a proposal to get the ball rolling toward a committable solution. Does it seem useful?

I think it could be useful. I see some useful overlap with the description design tool. It pre-populate the accessible name with something unique and human readable.

Upgrading to very useful.

Yes, cool idea! This seems great for internal use, I could see adding a query parameter that does this.
Thinking about the best place to put it, maybe somewhere in joist/SimDisplay? Like maybe we could override updateDisplay and add a walk down the AccessibleInstance tree to set these names?

Adding to the Monday board and scheduling for a future subteam sounds good.  Some questions for then:

Is there a way we can make this the default behavior, not behind a query parameter? For instance, if we automatically change the camel casing to spaced, like "Mystery Launcher 1" radio button instead of mysteryLauncher1 radio button, could it be good enough to use without a query parameter?  What else would need to be done before we make it available to screen reader users? Is it "something is better than nothing" or "we shouldn't add anything if we can't meet a certain quality/quantity threshold"?
I was unclear about the ParallelDOM lifecycle. I thought tapping in to the constructor or mutate would be the correct time to know that accessibleName is null but the tandem is specified. But I couldn't get that to work. But likely something along those lines could be helpful.
There are ideas at the top of the issue about using the translatable visually presented strings here. We could design a scaffolded system where: If the accessibleName is provided, it takes precedence. If not, check if the human visible string is accessible, such as a label on a button or next to a checkbox.  If not, check if the tandem was supplied, and convert it from camelcase to readable.

The alt-input management team discussed this today and we would like to loop this into the greater description work:
https://github.com/phetsims/joist/issues/941

Meeting 3/11/24

We need to explore how screen readers will treat camelCase names.
@terracoda mentioned that a name is better than no name.
It would most definitely be helpful at least in the a11y view. We can do this via a query parameter.
Further discussion on wether we want to publish these names with sims is needed.

@samreid @zepumph and I worked on this as part of https://github.com/phetsims/buoyancy/issues/109 and they wrote a patch for doing this automatically in some sun components. Leaving it here in case it's useful for future work

Subject: [PATCH] Pass launchButtonEnabledProperty directly to the launchButton.enabledProperty, see https://github.com/phetsims/projectile-data-lab/issues/141
---
Index: sun/js/Checkbox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/Checkbox.ts b/sun/js/Checkbox.ts
--- a/sun/js/Checkbox.ts	(revision 3cd5c702159e13688afd1b0b9da468f129ccee71)
+++ b/sun/js/Checkbox.ts	(date<PHONE_NUMBER>171)
@@ -121,6 +121,7 @@
       tagName: 'input',
       inputType: 'checkbox',
       appendDescription: true,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'Checkbox' ),

       // voicing
       voicingCheckedObjectResponse: null,
Index: sun/js/ComboBoxListItemNode.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/ComboBoxListItemNode.ts b/sun/js/ComboBoxListItemNode.ts
--- a/sun/js/ComboBoxListItemNode.ts	(revision 3cd5c702159e13688afd1b0b9da468f129ccee71)
+++ b/sun/js/ComboBoxListItemNode.ts	(date<PHONE_NUMBER>366)
@@ -69,6 +69,7 @@
       tagName: 'li',
       focusable: true,
       ariaRole: 'option',
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'Item' ),

       // the `li` with ariaRole `option` does not get click events on iOS VoiceOver, so position
       // elements so they receive pointer events
Index: sun/js/ComboBoxButton.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/ComboBoxButton.ts b/sun/js/ComboBoxButton.ts
--- a/sun/js/ComboBoxButton.ts	(revision 3cd5c702159e13688afd1b0b9da468f129ccee71)
+++ b/sun/js/ComboBoxButton.ts	(date<PHONE_NUMBER>495)
@@ -105,7 +105,8 @@
       // pdom
       containerTagName: 'div',
       labelTagName: 'p', // NOTE: A `span` causes duplicate name-speaking with VO+safari in https://github.com/phetsims/ratio-and-proportion/issues/532
-      accessibleNameBehavior: ACCESSIBLE_NAME_BEHAVIOR
+      accessibleNameBehavior: ACCESSIBLE_NAME_BEHAVIOR,
+      // accessibleName: Tandem.toAccessibleName( providedOptions, 'ComboBoxButton' )
     }, providedOptions );

     assert && assert( _.includes( ALIGN_VALUES, options.align ),
Index: sun/js/AccordionBox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/AccordionBox.ts b/sun/js/AccordionBox.ts
--- a/sun/js/AccordionBox.ts	(revision 3cd5c702159e13688afd1b0b9da468f129ccee71)
+++ b/sun/js/AccordionBox.ts	(date<PHONE_NUMBER>653)
@@ -198,6 +198,7 @@
       tagName: 'div',
       headingTagName: 'h3', // specify the heading that this AccordionBox will be, TODO: use this.headingLevel when no longer experimental https://github.com/phetsims/scenery/issues/855
       accessibleNameBehavior: AccordionBox.ACCORDION_BOX_ACCESSIBLE_NAME_BEHAVIOR,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'AccordionBox' ),

       // voicing
       voicingNameResponse: null,
Index: sun/js/ComboBox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/ComboBox.ts b/sun/js/ComboBox.ts
--- a/sun/js/ComboBox.ts	(revision 3cd5c702159e13688afd1b0b9da468f129ccee71)
+++ b/sun/js/ComboBox.ts	(date<PHONE_NUMBER>569)
@@ -244,6 +244,7 @@
       buttonLabelTagName: 'p',
       accessibleNameBehavior: ACCESSIBLE_NAME_BEHAVIOR,
       helpTextBehavior: HELP_TEXT_BEHAVIOR,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'ComboBox' ),

       comboBoxVoicingNameResponsePattern: SunConstants.VALUE_NAMED_PLACEHOLDER,
       comboBoxVoicingContextResponse: null,
@@ -565,6 +566,9 @@
       else if ( typeof item.a11yName === 'string' ) {
         property = new TinyProperty( item.a11yName );
       }
+      else if ( item.tandemName ) {
+        property = new TinyProperty( Tandem.tandemNameToAccessibleName( item.tandemName, 'Item' ) );
+      }
       else {
         property = new TinyProperty( null );
       }
Index: sun/js/AquaRadioButton.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/AquaRadioButton.ts b/sun/js/AquaRadioButton.ts
--- a/sun/js/AquaRadioButton.ts	(revision 3cd5c702159e13688afd1b0b9da468f129ccee71)
+++ b/sun/js/AquaRadioButton.ts	(date<PHONE_NUMBER>736)
@@ -119,7 +119,8 @@
       containerTagName: 'li',
       labelTagName: 'label',
       appendLabel: true,
-      appendDescription: true
+      appendDescription: true,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'RadioButton' )

     }, providedOptions );

Index: density-buoyancy-common/js/common/view/BuoyancyDisplayOptionsPanel.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/density-buoyancy-common/js/common/view/BuoyancyDisplayOptionsPanel.ts b/density-buoyancy-common/js/common/view/BuoyancyDisplayOptionsPanel.ts
--- a/density-buoyancy-common/js/common/view/BuoyancyDisplayOptionsPanel.ts	(revision f4549040101bae01a024fc86025259d7e038e20d)
+++ b/density-buoyancy-common/js/common/view/BuoyancyDisplayOptionsPanel.ts	(date<PHONE_NUMBER>354)
@@ -136,7 +136,7 @@
                   tandem: options.tandem.createTandem( 'massesCheckbox' )
                 }, checkboxOptions ) ),
                 new Checkbox( model.showForceValuesProperty, new Text( DensityBuoyancyCommonStrings.forceValuesStringProperty, labelOptions ), combineOptions<CheckboxOptions>( {
-                  tandem: options.tandem.createTandem( 'forcesCheckbox' )
+                  tandem: options.tandem.createTandem( 'forceValuesCheckbox' )
                 }, checkboxOptions ) ),
                 ...( model.supportsDepthLines ?
                   [ new Checkbox( model.showDepthLinesProperty, new Text( DensityBuoyancyCommonStrings.depthLinesStringProperty, labelOptions ), combineOptions<CheckboxOptions>( {
Index: tandem/js/Tandem.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/tandem/js/Tandem.ts b/tandem/js/Tandem.ts
--- a/tandem/js/Tandem.ts	(revision 27c3e2c9daa7165780d04465016c2e898a96c192)
+++ b/tandem/js/Tandem.ts	(date<PHONE_NUMBER>619)
@@ -11,9 +11,10 @@
 import arrayRemove from '../../phet-core/js/arrayRemove.js';
 import merge from '../../phet-core/js/merge.js';
 import optionize from '../../phet-core/js/optionize.js';
-import PhetioObject from './PhetioObject.js';
+import PhetioObject, { PhetioObjectOptions } from './PhetioObject.js';
 import TandemConstants, { PhetioID } from './TandemConstants.js';
 import tandemNamespace from './tandemNamespace.js';
+import PickOptional from '../../phet-core/js/types/PickOptional.js';

 // constants
 // Tandem can't depend on joist, so cannot use packageJSON module
@@ -597,6 +598,29 @@
    * Use this as the parent tandem for Properties that are related to sim-specific preferences.
    */
   public static readonly PREFERENCES = Tandem.GLOBAL_MODEL.createTandem( 'preferences' );
+
+  /**
+   * Tandem names can be used to create accessible names for screen readers. This method will convert a tandem name to
+   * a human-readable name. For example, 'resetAllButton' would become 'Reset All'.
+   */
+  public static toAccessibleName( providedOptions: PickOptional<PhetioObjectOptions, 'tandem'> | undefined, suffix: string ): string | null {
+    if ( providedOptions && providedOptions.tandem ) {
+      return Tandem.tandemNameToAccessibleName( providedOptions.tandem.name, suffix );
+    }
+    return null;
+  }
+
+  public static tandemNameToAccessibleName( tandemName: string, suffix: string ): string | null {
+    assert && assert( tandemName.toLowerCase().endsWith( suffix.toLowerCase() ), `suffix should be at the end of the tandem name: ${tandemName}` );
+
+    // trim the suffix
+    const withoutSuffix = tandemName.slice( 0, -suffix.length );
+
+    const whitespaceName = withoutSuffix.replace( /([A-Z])/g, ' $1' ).trim();
+
+    // capitalize the first letter of each word, no matter how many words
+    return whitespaceName.replace( /\b\w/g, c => c.toUpperCase() );
+  }
 }

 Tandem.addLaunchListener( () => {

In addition to the above, it does seem that Tandem.toAccessibleName() is getting used in RectangularRadioButton, but that is all. I'd like to take a closer look.

It seems awesome to try to add this eagerly
The components we add this to need to get audited to make sure they aren't also sometimes getting their accessibleName from some other strategy in certain cases.

I really would like to apply some form of this patch, but there are too many cases where common code is using the older style of labelContent/innerContent for the accessible name. We can't just add accessibleName as the default, because it doesn't get overwritten in a standard way. My guess is that I could get to a commit point in 2 hours, but it would involve making pretty error prone changes to get more consistent to having common code set default accessible names.

Subject: [PATCH] Pass launchButtonEnabledProperty directly to the launchButton.enabledProperty, see https://github.com/phetsims/projectile-data-lab/issues/141
---
Index: area-model-common/js/common/view/AreaScreenView.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/area-model-common/js/common/view/AreaScreenView.js b/area-model-common/js/common/view/AreaScreenView.js
--- a/area-model-common/js/common/view/AreaScreenView.js	(revision 1241c8db48406403bb2f81ad204a2d47355911b1)
+++ b/area-model-common/js/common/view/AreaScreenView.js	(date<PHONE_NUMBER>680)
@@ -137,8 +137,7 @@
         contentYSpacing: model.allowExponents ? 5 : 8,

         // pdom
-        labelTagName: 'h3',
-        labelContent: AreaModelCommonStrings.a11y.factorsBoxStringProperty,
+        accessibleName: AreaModelCommonStrings.a11y.factorsBoxStringProperty,
         titleBarOptions: {
           descriptionContent: AreaModelCommonStrings.a11y.factorsBoxDescriptionStringProperty
         }
Index: masses-and-springs/js/common/view/SpringScreenView.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/masses-and-springs/js/common/view/SpringScreenView.js b/masses-and-springs/js/common/view/SpringScreenView.js
--- a/masses-and-springs/js/common/view/SpringScreenView.js	(revision 927626152eaf1c63178fe4006b8ecf636164adf5)
+++ b/masses-and-springs/js/common/view/SpringScreenView.js	(date<PHONE_NUMBER>019)
@@ -273,7 +273,7 @@
    */
   createStopperButton( spring, tandem ) {
     return new StopperButtonNode(
-      tandem.createTandem( 'secondSpringStopperButtonNode' ), {
+      tandem.createTandem( 'secondSpringStopperButton' ), {
         listener: () => {
           spring.stopSpring();
         },
Index: sun/js/AquaRadioButton.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/AquaRadioButton.ts b/sun/js/AquaRadioButton.ts
--- a/sun/js/AquaRadioButton.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/AquaRadioButton.ts	(date<PHONE_NUMBER>681)
@@ -119,10 +119,14 @@
       containerTagName: 'li',
       labelTagName: 'label',
       appendLabel: true,
-      appendDescription: true
+      appendDescription: true,
+      labelContent: Tandem.toAccessibleName( providedOptions, 'RadioButton' )

     }, providedOptions );

+    if ( options.hasOwnProperty( 'accessibleName' ) || options.hasOwnProperty( 'innerContent' ) ) {
+      throw new Error( 'inner content for aquaRadioButton' );
+    }
     super();

     this.value = value;
Index: scenery-phet/js/buttons/ResetAllButton.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/scenery-phet/js/buttons/ResetAllButton.ts b/scenery-phet/js/buttons/ResetAllButton.ts
--- a/scenery-phet/js/buttons/ResetAllButton.ts	(revision e1d063709def87011597a2edcaf12a6f465efc9c)
+++ b/scenery-phet/js/buttons/ResetAllButton.ts	(date<PHONE_NUMBER>491)
@@ -61,7 +61,7 @@
       soundPlayer: resetAllSoundPlayer,

       // pdom
-      innerContent: SceneryPhetStrings.a11y.resetAll.labelStringProperty,
+      accessibleName: SceneryPhetStrings.a11y.resetAll.labelStringProperty,

       // voicing
       voicingNameResponse: SceneryPhetStrings.a11y.resetAll.labelStringProperty,
Index: tandem/js/Tandem.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/tandem/js/Tandem.ts b/tandem/js/Tandem.ts
--- a/tandem/js/Tandem.ts	(revision eb8d2f1587c2bb63481ca4cec19c6d27bfc1464b)
+++ b/tandem/js/Tandem.ts	(date<PHONE_NUMBER>660)
@@ -614,10 +614,12 @@
    * a human-readable name. For example, 'resetAllButton' would become 'Reset All'.
    */
   public static tandemNameToAccessibleName( tandemName: string, suffix: string ): string | null {
-    assert && assert( tandemName.toLowerCase().endsWith( suffix.toLowerCase() ), `suffix should be at the end of the tandem name: ${tandemName}` );

     // trim the suffix
-    const withoutSuffix = tandemName.slice( 0, -suffix.length );
+    let withoutSuffix = tandemName;
+    if ( tandemName.toLowerCase().endsWith( suffix.toLowerCase() ) ) {
+      withoutSuffix = tandemName.slice( 0, -suffix.length );
+    }

     const whitespaceName = withoutSuffix.replace( /([A-Z])/g, ' $1' ).trim();

Index: sun/js/buttons/RectangularRadioButton.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/buttons/RectangularRadioButton.ts b/sun/js/buttons/RectangularRadioButton.ts
--- a/sun/js/buttons/RectangularRadioButton.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/buttons/RectangularRadioButton.ts	(date<PHONE_NUMBER>461)
@@ -94,7 +94,7 @@
       containerTagName: 'li',
       appendDescription: true,
       appendLabel: true,
-      accessibleName: Tandem.toAccessibleName( providedOptions, 'RadioButton' ),
+      labelContent: Tandem.toAccessibleName( providedOptions, 'RadioButton' ),

       // phet-io
       tandem: Tandem.REQUIRED,
Index: sun/js/ComboBox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/ComboBox.ts b/sun/js/ComboBox.ts
--- a/sun/js/ComboBox.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/ComboBox.ts	(date<PHONE_NUMBER>006)
@@ -244,6 +244,7 @@
       buttonLabelTagName: 'p',
       accessibleNameBehavior: ACCESSIBLE_NAME_BEHAVIOR,
       helpTextBehavior: HELP_TEXT_BEHAVIOR,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'ComboBox' ),

       comboBoxVoicingNameResponsePattern: SunConstants.VALUE_NAMED_PLACEHOLDER,
       comboBoxVoicingContextResponse: null,
@@ -259,6 +260,9 @@
       phetioEnabledPropertyInstrumented: true // opt into default PhET-iO instrumented enabledProperty
     }, providedOptions );

+    if ( options.hasOwnProperty( 'labelContent' ) || options.hasOwnProperty( 'innerContent' ) ) {
+      throw new Error( 'inner content for combo' );
+    }
     const nodes = getGroupItemNodes( items, options.tandem.createTandem( 'items' ) );

     assert && nodes.forEach( node => {
@@ -561,6 +565,9 @@
       else if ( typeof item.a11yName === 'string' ) {
         property = new TinyProperty( item.a11yName );
       }
+      else if ( item.tandemName ) {
+        property = new TinyProperty( Tandem.tandemNameToAccessibleName( item.tandemName, 'Item' ) );
+      }
       else {
         property = new TinyProperty( null );
       }
Index: sun/js/buttons/TextPushButton.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/buttons/TextPushButton.ts b/sun/js/buttons/TextPushButton.ts
--- a/sun/js/buttons/TextPushButton.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/buttons/TextPushButton.ts	(date<PHONE_NUMBER>232)
@@ -38,7 +38,7 @@

       // RectangularPushButtonOptions
       tandem: Tandem.REQUIRED,
-      innerContent: string
+      accessibleName: string
     }, providedOptions );

     const text = new Text( string, combineOptions<TextOptions>( {
Index: sun/js/buttons/ButtonNode.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/buttons/ButtonNode.ts b/sun/js/buttons/ButtonNode.ts
--- a/sun/js/buttons/ButtonNode.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/buttons/ButtonNode.ts	(date<PHONE_NUMBER>827)
@@ -23,6 +23,8 @@
 import TButtonAppearanceStrategy, { TButtonAppearanceStrategyOptions } from './TButtonAppearanceStrategy.js';
 import TContentAppearanceStrategy, { TContentAppearanceStrategyOptions } from './TContentAppearanceStrategy.js';
 import TinyProperty from '../../../axon/js/TinyProperty.js';
+import Tandem from '../../../tandem/js/Tandem.js';
+import inheritance from '../../../phet-core/js/inheritance.js';

 // constants
 const CONTRAST_FILTER = new Contrast( 0.7 );
@@ -123,6 +125,8 @@
       buttonAppearanceStrategyOptions: {},
       contentAppearanceStrategy: null,
       contentAppearanceStrategyOptions: {},
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'Button' ),
+
       enabledAppearanceStrategy: ( enabled, button, background, content ) => {
         background.filters = enabled ? [] : [ CONTRAST_FILTER, BRIGHTNESS_FILTER ];

@@ -151,6 +155,9 @@
     options.enabledProperty = buttonModel.enabledProperty;

     super();
+    if ( ( inheritance( this ).map( x => x.name ).includes( 'RectangularRadioButton' ) && options.hasOwnProperty( 'labelContent' ) ) || options.hasOwnProperty( 'innerContent' ) ) {
+      throw new Error( 'inner content for button' );
+    }

     this.content = options.content;
     this.buttonModel = buttonModel;
Index: sun/js/ComboBoxListItemNode.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/ComboBoxListItemNode.ts b/sun/js/ComboBoxListItemNode.ts
--- a/sun/js/ComboBoxListItemNode.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/ComboBoxListItemNode.ts	(date<PHONE_NUMBER>011)
@@ -69,6 +69,7 @@
       tagName: 'li',
       focusable: true,
       ariaRole: 'option',
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'Item' ),

       // the `li` with ariaRole `option` does not get click events on iOS VoiceOver, so position
       // elements so they receive pointer events
@@ -89,6 +90,9 @@
       visiblePropertyOptions: { phetioFeatured: true }
     }, providedOptions );

+    if ( options.hasOwnProperty( 'labelContent' ) || options.hasOwnProperty( 'innerContent' ) ) {
+      throw new Error( 'inner content for combobox button' );
+    }
     // @ts-expect-error convert Property into string
     options.comboBoxVoicingNameResponsePattern = options.comboBoxVoicingNameResponsePattern.get ?
       // @ts-expect-error convert Property into string
Index: sun/js/Checkbox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/Checkbox.ts b/sun/js/Checkbox.ts
--- a/sun/js/Checkbox.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/Checkbox.ts	(date<PHONE_NUMBER>021)
@@ -121,6 +121,7 @@
       tagName: 'input',
       inputType: 'checkbox',
       appendDescription: true,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'Checkbox' ),

       // voicing
       voicingCheckedObjectResponse: null,
@@ -133,6 +134,9 @@
       voiceNameResponseOnSelection: true
     }, providedOptions );

+    if ( options.hasOwnProperty( 'labelContent' ) || options.hasOwnProperty( 'innerContent' ) ) {
+      throw new Error( 'inner content for checkbox' );
+    }
     super();

     // sends out notifications when the checkbox is toggled.
Index: sun/js/AccordionBox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/sun/js/AccordionBox.ts b/sun/js/AccordionBox.ts
--- a/sun/js/AccordionBox.ts	(revision eec0449cc1afd5b7885682250a167ae4d472e700)
+++ b/sun/js/AccordionBox.ts	(date<PHONE_NUMBER>998)
@@ -198,6 +198,7 @@
       tagName: 'div',
       headingTagName: 'h3', // specify the heading that this AccordionBox will be, TODO: use this.headingLevel when no longer experimental https://github.com/phetsims/scenery/issues/855
       accessibleNameBehavior: AccordionBox.ACCORDION_BOX_ACCESSIBLE_NAME_BEHAVIOR,
+      accessibleName: Tandem.toAccessibleName( providedOptions, 'AccordionBox' ),

       // voicing
       voicingNameResponse: null,
@@ -218,6 +219,9 @@
       }
     }, providedOptions );

+    if ( options.hasOwnProperty( 'labelContent' ) || options.hasOwnProperty( 'innerContent' ) ) {
+      throw new Error( 'inner content for accordionbox' );
+    }
     // expandCollapseButtonOptions defaults
     options.expandCollapseButtonOptions = combineOptions<ExpandCollapseButtonOptions>( {
       sideLength: 16, // button is a square, this is the length of one side

Concerns about "discovering" accessible names from tandem names, and specific problems caused in https://github.com/phetsims/sun/issues/904 are noted in https://github.com/phetsims/sun/issues/904.

Here is an untested idea for discovering the translatable text for a checkbox:

Subject: [PATCH] Pass repo to check, see https://github.com/phetsims/perennial/issues/364
---
Index: js/Checkbox.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/Checkbox.ts b/js/Checkbox.ts
--- a/js/Checkbox.ts	(revision 1e2dd5869880fd41672adeeba92aa300f40b410e)
+++ b/js/Checkbox.ts	(date<PHONE_NUMBER>066)
@@ -14,7 +14,7 @@
 import InstanceRegistry from '../../phet-core/js/documentation/InstanceRegistry.js';
 import optionize from '../../phet-core/js/optionize.js';
 import StrictOmit from '../../phet-core/js/types/StrictOmit.js';
-import { assertNoAdditionalChildren, FireListener, LayoutConstraint, Node, NodeOptions, Path, Rectangle, SceneryConstants, TPaint, Voicing, VoicingOptions, WidthSizable, WidthSizableOptions } from '../../scenery/js/imports.js';
+import { assertNoAdditionalChildren, FireListener, LayoutConstraint, Node, NodeOptions, Path, Rectangle, RichText, Text, SceneryConstants, TPaint, Voicing, VoicingOptions, WidthSizable, WidthSizableOptions } from '../../scenery/js/imports.js';
 import checkEmptySolidShape from '../../sherpa/js/fontawesome-4/checkEmptySolidShape.js';
 import checkSquareOSolidShape from '../../sherpa/js/fontawesome-4/checkSquareOSolidShape.js';
 import sharedSoundPlayers from '../../tambo/js/sharedSoundPlayers.js';
@@ -25,6 +25,7 @@
 import Tandem from '../../tandem/js/Tandem.js';
 import Utterance, { TAlertable } from '../../utterance-queue/js/Utterance.js';
 import sun from './sun.js';
+import TReadOnlyProperty from '../../axon/js/TReadOnlyProperty.js';

 // constants
 const BOOLEAN_VALIDATOR = { valueType: 'boolean' };
@@ -86,6 +87,26 @@

   public constructor( property: PhetioProperty<boolean>, content: Node, providedOptions?: CheckboxOptions ) {

+    // recursive search over content.children to find any Text
+    // if found, set the accessibleName of the Checkbox to the text of the first Text found
+    const findText = ( node: Node ): string | null | TReadOnlyProperty<string> => {
+      if ( node instanceof Text ) {
+        return node.stringProperty;
+      }
+      else if ( node instanceof RichText ) {
+        return node.stringProperty;
+      }
+      else if ( node.children ) {
+        for ( const child of node.children ) {
+          const text = findText( child );
+          if ( text ) {
+            return text;
+          }
+        }
+      }
+      return null;
+    };
+
     const options = optionize<CheckboxOptions, SelfOptions, ParentOptions>()( {

       // CheckboxOptions
@@ -129,7 +150,9 @@
       // the voicingContextResponse
       checkedContextResponse: null,
       uncheckedContextResponse: null,
-      voiceNameResponseOnSelection: true
+      voiceNameResponseOnSelection: true,
+
+      accessibleName: findText( content )
     }, providedOptions );

     super();

In dev meeting today, @pixelzoom agreed to move forward with the tandem names shown as in the lower picture in https://github.com/phetsims/scenery/issues/1526#issuecomment-1944166933

My input is that we should develop reusable strategies for discovering acessibleName, and use them in the appropriate UI components.  For example, "look for string Property in content Node" seems like a strategy (and implementation) that would be identical for many UI components.

Meeting 11/1/24

we would like to prioritize this on a component by component basis
Any accessibleName that is "discovered" by the code should be the default, but needs to have the ability to easily be overwritten.
Assigning to @jessegreenberg for next steps.

Thanks @samreid , https://github.com/phetsims/scenery/issues/1526#issuecomment-2432602989 is a good idea!
I just committed an implementation in https://github.com/phetsims/scenery/commit/8b34da45b0a27f34f776f39169e61bf9e872e1ae.
@pixelzoom would you mind reviewing this? It is being used in Checkbox and Dialog now.

https://github.com/phetsims/scenery/commit/8b34da45b0a27f34f776f39169e61bf9e872e1ae looks good to me. PDOMUtils seems like the right place for this, and findStringProperty seems like a good name.
The usage in Checkbox https://github.com/phetsims/sun/commit/2cf8d3d9259185e20e5b08967037070d33762ae8 also looks nice.
Back to @jessegreenberg for next steps.

Great, thank you for reviewing. I created https://github.com/phetsims/sun/issues/911, where we will track rolling it out by using it in PhET's common code components. Closing this issue.


find_packages() doesn't find PEP 420 packages

Originally reported by: gwideman (Bitbucket: gwideman, GitHub: gwideman)

#!python

setup(...
    packages=find_packages(..)
    )

on the developer machine will fail to find packages that lack a __init__.py file, as is allowed in Python 3.3. However, such packages listed explicitly: packages=['mypkg'] do appear to get included and later installed.
Note: When testing this, before each test be sure to delete all generated metadata, including that which setup may previously have placed in the original source directory, as it seems that setup may use metadata created on a previous run in order to include files.
This is part of a general problem reported in issue #83, but I've logged it separately as it's specifically about setuptools.

Bitbucket: https://bitbucket.org/pypa/setuptools/issue/97

Another issue that's come up recently on distutils-sig is that nspkg.pth support is broken for PEP 420 currently -- the magic really shouldn't be running if PEP 420 support is available.

This issue has since been addressed in #805.

That's why it would be simplest to change setuptools.dist:check_nsp to automatically add the namespace packages to the list of packages, instead of giving an error -- at that point, nobody should even notice that find_packages() isn't finding the packages in question. (Though the docs should still be updated to clarify that any init-less packages must still be manually listed in namespace_packages.)

This sounds like a fine recommendation, which I will attempt to implement.

In the issue-97 branch, I've attempted to implement the recommendation, but I'm pretty sure this doesn't solve the problem. In particular, it still doesn't address the issue of a very basic use case like zope.interface. If zope/__init__.py does not exist, setuptools.find_packages() will still not find zope/interface/__init__.py.

Anyone know if there has been any more work on this? I'm currently in the situation of wanting to switch from using an "old school" namespace package to a PEP-420 one. In my case I have only a single (company-wide) namespace package, and various distributions that install packages into it, so for me, the solution @jaraco was working on in the issue-97 branch would be great, if only find_packages was updated as well, such that it also looks at namespace_packages when taking the decision on whether it should enter a directory in its search.
What I mean is, if find_packages encounters:
some_project/setup.py
some_project/my_company/foo/__init__.py
some_project/my_company/foo/stuff.py
some_project/my_company/bar/__init__.py
some_project/my_company/bar/morestuff.py

Then find_packages would normally not enter into my_company (the PEP-420 ns package). But if it was changed to also look at namespace_packages, it would enter, because I have my_company in there (I already do that). @jaraco: Would something like that work or am I babbling? Did you have any plans like that for your branch?
I would be perfectly fine with having to explicitly specify all my namespace packages in namespace_packages (in fact, I already do that with this single ns package that I have). Having find_packages work would still be a great boon compared to explicitly having to list all packages.

My recommendation for now is that package maintainers continue to use __init__.py files in their namespace packages to mark the folder as a package for find_packages. Either that or enumerate the packages explicitly and not use find_packages. I haven't made any advancement on the branch, but I do agree, what you propose would be preferable.
I've flagged this issue as help-wanted because I simply don't have time to address all of these issues on my own. Please feel free to grab the work I've done and advance it, or start again from the current master.

@jaraco Alright, I realized that what I said earlier doesn't really make sense, since find_package has no way of getting a hold of what was passed in to namespace_packages (right?).
We're moving to PEP 420, so what I've done for now is to keep a setup_helper.py next to my setup.py with:
from distutils.util import convert_path
from fnmatch import fnmatchcase

import os

def find_packages(where='.', exclude=(), include=('*',), pep420_dirs=()):
    """Our own version of the setuptools find_package helper

    The additional 'pep420_dirs' parameter specifies directories which are
    PEP 420 packages. These will be considered packages despite lacking a
    __init__.py. We use this until setuptools issue #97 is fixed.
    """
    where = convert_path(where)
    exclude += ('*__pycache__',)
    packages = []
    for root, dirs, files in os.walk(where, followlinks=True):
        all_dirs = dirs[:]
        dirs[:] = []
        for dir in all_dirs:
            full_path = os.path.join(root, dir)
            rel_path = os.path.relpath(full_path, where)
            package = rel_path.replace(os.path.sep, '.')
            included = any(fnmatchcase(package, pat=pat) for pat in include)
            excluded = any(fnmatchcase(package, pat=pat) for pat in exclude)
            has_init = os.path.isfile(os.path.join(full_path, '__init__.py'))
            if ('.' in dir) or not (rel_path in pep420_dirs or has_init):
                continue
            if included and not excluded:
                packages.append(package)
            dirs.append(dir)
    return packages

and then I do
    packages=find_packages(pep420_dirs=['my_company']),
    namespace_packages=['my_company'],

in the setup call. It's not too pretty, but works well enough until there's a better solution.

@jaraco I haven't hacked on setuptools before, but I could give it a shot.
Though, the only way I can think of to make the find_packages search be guided by both presence of __init__.py and namespace_packages is to somehow delay the entire search until namespace_packages is known.
So find_packages would do nothing but store away the fact that it has been called, its parameters, and that the search is still a TODO. And then at some later stage the actual search is done. But this would break the use of find_packages in any other context except for the call to setup(..) :(
Do you see some other way?
Perhaps find_packages should be left completely alone, and some sentient value should instead be stored, e.g. packages=RequestFindPackages(...), and then at a later stage, if isinstance(packages, RequestFindPackages) the search is performed (obviously with some better name...).

@jaraco One simple solution that I kind of like, to avoid having to defer the search until namespace_packages is known, would be to change the behavior of include such that a directory in include is considered a package despite lacking __init__.py, and then instruct users to add their PEP 420 packages to include.
So it would be something like
implicit_namespace_packages=[...]

setup(
    ...
    packages=find_packages(include=implicit_namespace_packages, ...)
    namespace_packages=[...] + implicit_namespace_packages
)

I know it requires the user to introduce a variable to avoid having to type the list of implicit namespace packages twice, but I could live with that. For many use cases (e.g. a "suite" of distributions) I think the list of PEP 420 packages should be short (often just a single one), and the user might then even skip the variable.
I think it makes sense that if we've now discovered that find_packages needs more info to do its job, that the info must be passed in as a parameter.
What do you think?

Regarding the first idea (deferring the search until namespace_packages is known), we could of course just do the search over again, but this time guided by namespace_packages, and during the deferred search only add packages which are not already added. A little more inefficient, but would preserve the behavior of find_packages for those who use it outside of setup(...).

Yet another crazy idea: We could allow dict entries in packages, to signify "search requests":
setup(
    packages=[
        'some_hardcoded_package',
        { 'exclude': '*.tests' }
    ],
    namespace_packages=['my_company']

The dicts would support the normal find_packages parameters as keys, and would be expanded to the result of the search at a later time when namespace_packages is known. It's a little funky, but I think I like it.

Of my ideas so far, the one that is the least amount of code is to change the meaning of include. This is what it would look like:
diff --git a/setuptools/__init__.py b/setuptools/__init__.py
index 54577ce..ac7f524 100644
--- a/setuptools/__init__.py
+++ b/setuptools/__init__.py
@@ -52,18 +52,24 @@ class PackageFinder(object):
         specified, only the named packages will be included.  If it's not
         specified, all found packages will be included.  'include' can contain
         shell style wildcard patterns just like 'exclude'.
+
+        Note: PEP 420 packages will not be included unless they are explicitly
+              listed with their full names in 'include', so if you use PEP 420
+              you would use something like find_packages(include=('*', 'foo'))
+              if foo is a PEP 420 package.
         """

         return list(cls._find_packages_iter(
             convert_path(where),
             cls._build_filter('ez_setup', '*__pycache__', *exclude),
-            cls._build_filter(*include)))
+            cls._build_filter(*include), include))

     @classmethod
-    def _find_packages_iter(cls, where, exclude, include):
+    def _find_packages_iter(cls, where, excluded, included, include):
         """
-        All the packages found in 'where' that pass the 'include' filter, but
-        not the 'exclude' filter.
+        All the packages found in 'where' that pass the 'included' filter, but
+        not the 'excluded' filter. If a package is in 'include', it's considered
+        a package even if it lacks an __init__.py.
         """
         for root, dirs, files in os.walk(where, followlinks=True):
             # Copy dirs to iterate over it, then empty dirs.
@@ -76,11 +82,11 @@ class PackageFinder(object):
                 package = rel_path.replace(os.path.sep, '.')

                 # Skip directory trees that are not valid packages
-                if ('.' in dir or not cls._looks_like_package(full_path)):
+                if ('.' in dir or (not cls._looks_like_package(full_path) and package not in include)):
                     continue

                 # Should this package be included?
-                if include(package) and not exclude(package):
+                if included(package) and not excluded(package):
                     yield package

                 # Keep searching subdirectories, as there may be more packages

And then e.g.
setup(
    packages=find_packages(include=['*', 'my_company']),
    namespace_packages('my_company')
)

I realize I've had a misconception about one thing in my thinking: I thought that you still had to list PEP 420 packages in the namespace_packages parameter to setup. But I realize now that you only need to do this for old style namespace packages (those with __init__.py). So ignore all my talk about trying to avoid repetition.
In light of that, I think the absolutely best/easiest solution is to simply add an implicit_namespace_packages parameter to find_packages, where the use must list any PEP 420 packages explicitly. I'll prepare a PR to do so.

Hello.
I made a small patch here with something that works for me, but needs a bit more work after discussion: https://github.com/pypa/setuptools/pull/943
This is a solution so that the simple case works without requiring any extra parameters be used.
I modified sampleproject here: https://github.com/illume/sampleproject
Note how there is only a file sample/sample.py and no __init__.py.
This is the simplest possible package. I think this is a very important use case for many people, and I don't think find_packages() should require doing anything special with any extra parameters for this case. I've seen many people trip over this issue, since it's pretty much the simplest type of package you can make.
It works for me on the sampleproject above, and some other projects I tested. I handled the case where it ignores directories with site-packages in them (probably a venv, definitely not a package).
 Also, I need to make it only look for namespace packages in python 3.3+
What do you think?

@illume I think if you read the discussion above, there were concerns that simply using "if a dir has any .py files below it, it's a package" as the guiding logic in the search would make it too liberal for many projects.

Thanks @estan for your PR. It's a thorough implementation for our consideration. I'm considering it, but it does have a couple of smells that I'd like to avoid, namely that it passes a parameter through several calls, but also because that parameter has to be supplied twice (for packages and namespace_packages).
Someone recently suggested a different approach - using a sentinel file to signal a namespace package. Consider, for example, allowing a __nspkg__ 0-length file (or similar) to declare a directory to be a namespace package.
Such an approach would have the simple elegance of being stored with the source control system and being declarative. It would have the additional benefit of enabling the replacing the packages and namespace_packages parameters to discover these packages, such that every setup.py might look simply like:
setup(
  packages=find_packages(),
  namespace_packages=find_namespace_packages(),
)

What do the subscribers of this ticket think of such an idea?
@ncoghlan: How does such a proposal strike you? My instinct is that such a convention should be more widely reviewed.

How about a find_packages that works correctly on python 3.3+, but falls back to current behaviour for older pythons? Legacy find_packages can still work as is for older packages.
The problem with adding a separate namespace_packages function, or a separate argument is that I don't think people will know to look for them. How do users figure out the reason their package that works in python doesn't work when packaged?
Common folders like doc, and virtualenv ones can be removed from consideration by find_packages by looking for things like 'site-packages' and common sphinx files. Note, that we already have an exclude argument in find_packages.
Is there a tool to test setuptools changes over a wide range of packages already? I would be nice to base decisions on data. I'd be happy to do the work to find out if filtering out false positives will be effective.
When people have a sample/sample.py which works as a package in python, and then doesn't with packaging, we have a major packaging wart.
If find_packages is not going to be fixed, perhaps we can stop recommending to use it in the docs, and instead recommend a 'new_find_packages' or whatever a new api could be called?

@jaraco: I was under the impression that in current versions, you don't have to list implicit namespace packages in namespace_packages (the setup parameter). So there would be no duplication, right? My new parameter would be the only place where you have to list them. But I may have misunderstood that.
That said, I really like the approach with __nspkg__ markers that you describe.
@illume: "that works correctly on python 3.3+" <-- I think that trying to figure out what this means is what has made this issue so longstanding. People seem to think that taking all directories which have .py files below them as implicit namespace packages is a too liberal approach, even if the user can exclude things, and even if there are a set of default exclusions. If you look back in this thread you'll see that it has already been discussed.

@jaraco: Regarding "passing a parameter through several calls", do you mean that namespace_packages is passed on from find to find_packages_iter? If so, how else should it be implemented?

@estan I don't think a broken by default find_packages is good, even if people have discussed that before (yes, I did read the previous discussion). The accepted PEP 420 now says that packages are directories with .py files in them. Rather than dismiss the approach I suggested out of hand I'd like to look into it to see if it is a real problem. I haven't been able to find a testing tool for setuptools, so it looks like I'll have to hack something up.
When I help people with packaging this is one of the main issues that keeps coming up over and over again, and I doubt having a separate argument or function will help. Packaging in python is already arcane enough without adding another parameter or function call you'd need to add to make simple packages work.

@illume: Yes, personally I actually do think that approach could be viable. Sorry if I sounded dismissive.
Lets try to investigate what the impact would be for existing projects. Of the 100 most downloaded projects on PyPI, here are the 29 of them that uses find_packages and how they call it:
amqp-2.1.4 find_packages(exclude=['ez_setup', 't', 't.*'])
anyjson-0.3.3 find_packages(exclude=['ez_setup', 'examples', 'tests'])
awscli-1.11.44 find_packages(exclude=['tests*'])
beautifulsoup4-4.5.3 find_packages(exclude=['tests*'])
billiard-<IP_ADDRESS> find_packages(exclude=['ez_setup', 't', 't.*'])
botocore-1.5.7 find_packages(exclude=['tests*'])
celery-4.0.2 find_packages(exclude=['t', 't.*'])
cryptography-1.7.2 find_packages(where="src", exclude=["_cffi_src", "_cffi_src.*"])
Django-1.10.5 find_packages(exclude=['django.conf.project_template', 'django.conf.app_template', 'django.bin'])
Fabric-1.13.1 find_packages()
gevent-1.2.1 find_packages('src')
gunicorn-19.6.0 find_packages(exclude=['examples', 'tests'])
jmespath-0.9.1 find_packages(exclude=['tests'])
kombu-4.0.2 find_packages(exclude=['t', 't.*'])
logilab-common-1.3.0 find_packages(exclude=['contrib', 'docs', 'test*'])
Mako-1.0.6 find_packages('.', exclude=['examples*', 'test*'])
nose-1.3.7 find_packages()
Paste-2.0.3 find_packages(exclude=['ez_setup', 'examples', 'packages', 'tests*'])
PasteDeploy-1.5.2 find_packages(exclude=['tests'])
Pillow-4.0.0 find_packages()
pip-9.0.1 find_packages(exclude=["contrib", "docs", "tests*", "tasks"])
Pygments-2.2.0 find_packages()
pyOpenSSL-16.2.0 find_packages(where="src")
setuptools-34.1.0 find_packages(exclude=['*.tests'])
setuptools-git-1.1 find_packages()
Sphinx-1.5.2 find_packages(exclude=['tests'])
SQLAlchemy-1.1.5 find_packages('lib')
supervisor-3.3.1 find_packages()
zope.interface-4.3.3 find_packages('src')

It would be interesting to see how an "any dir with a .py below it" approach would compare to the current behavior for these projects. I might script this and check when I have the time.

Consider allowing a nspkg 0-length file to declare a directory to be a namespace package.

It could even be named __init__.py!
I am hearing some support for the idea that namespace packages should be marked in some way. Frankly I consider that to be a bare minimum requirement to take packages seriously.  It makes no sense to me that a package is in "the eye of the beholder".  The developer who places python files in a directory must surely know whether they are developing a package or just a bunch of files.  Furthermore, their development in that directory is predicated on it being or not being a package.
Yet if the directory is not marked as a package, then human readers don't know how to use it, and as we've seen in this now three year thread, automated tools don't know what to do with it either.
A package is a fundamental unit of code with which one builds (not to mention deploys) applications, and yet somehow python lacks a way to intentionally establish where one is?   There's much to love about python, but this incapacity to identify what is a package is ridiculous.
Serious question:   Is there any difference in actually accessing a package (via import etc) between an __init__.py package and a namespace package?   Maybe there's a difference of intent. And obviously one has an __init__.py file that may limit access to the enclosed modules and objects.  But if you use an __init__.py file that gives access to all enclosed, this would be imported and called just like the same package with no __init__.py file (ie: a namespace package), no?   Why are there even these two concepts?
Maybe I'm missing something, but it just seems to me that namespace packages need to be rethought.

@gwideman: I agree with pretty much everything you say. I think that PEP 420 is not well thought out, especially for tools needing to deduce package structure from the file system (or vice versa actually, in the case of py.test's automatic PYTHONPATH-prepending during py.test some_dir...).
In my case, my reason for moving from a classical namespace package for our company stuff (spread across various distributions) to a PEP 420 was this: I had made an internal dev tool, which I packaged as an Ubuntu package and uploaded to our company internal APT repo, and since the debhelper for Python distributions actually strips the __init__.py for namespace packages that are declared as such with namespace_packages in the setup call (wanting to encourage PEP 420 I guess, and avoid file conflicts), the namespace package was in fact turned into a PEP 420 one. The result was that developers working on other distributions of ours, with packages in that same namespace package, who set up their PYTHONPATH to point at the distribution they were hacking on, got the dev tool broken since Python couldn't see its PEP 420 package (the classical one in PYTHONPATH clouded it)... So that's my story on why I tried moving all our packages to PEP 420 and finding this issue #97.

@gwideman: I think that @jaraco meant that the __nspkg__ would only be present in the package dir in the distribution, not installed when doing setup.py install, so just a marker for the purpose of getting find_packages working (and possibly other tools), right @jaraco?
If it's to also be installed, then I think distro packagers will start yelling because it's yet another thing that will cause conflicts (in addition to classical __init__.py carrying namespace packages)..

@estan cool! How did you get those packages? Is there a tool of some sort to download them? I noticed there is a tool called Basket, and the list of top downloaded packages from https://python3wos.appspot.com/ Are these the correct places? Also, I'll grab a list of the 40 most recently updates packages as per https://pypi.python.org/pypi . I'll make a vagrant script to do it all inside.
I was thinking to modify find_packages to call both methods, and log (package_name, old_find_packages, new_packages, are_different). So then I can investigate the packages with differences (extra unwanted dirs).

Regarding "passing a parameter through several calls", do you mean that namespace_packages is passed on from find to find_packages_iter? If so, how else should it be implemented?

Indeed, I don't see a better way. And perhaps I'm being too sensitive here. I'll drop that concern.

I was under the impression that in current versions, you don't have to list implicit namespace packages in namespace_packages (the setup parameter). So there would be no duplication, right?

You may be right, though I haven't looked into it deeply. I do know that if you want the __init__.py file for backward compatibility, you do need to declare the namespace package in setup.py. The tools do need some way to know all namespace packages in order to generate the appropriate -nspkg.pth files on older Pythons, but for pure PEP 420 packages, that parameter may be unnecessary.

It [__nspkg__] could even be named init.py!

It can't because the presence of a __init__.py precludes the initialization or inclusion of that package as a namespace package.

Serious question: Is there any difference in actually accessing a package (via import etc) between an init.py package and a namespace package? Maybe there's a difference of intent. And obviously one has an init.py file that may limit access to the enclosed modules and objects. But if you use an init.py file that gives access to all enclosed, this would be imported and called just like the same package with no init.py file (ie: a namespace package), no? Why are there even these two concepts?

Yes. There's some good reading in pep 402 and the second paragraph of the PEP 420 spec specifically says namespace packages cannot contain a __init__.py. This behavior allows a package to be a "regular" package and exclude the possibility of other modules appearing in the same namespace.

Maybe I'm missing something, but it just seems to me that namespace packages need to be rethought.

For better or worse, this design is the one we must work with in the build tools. The design won't change until at least Python 3.7, and before that happens, the issue will need to be well-understood and a cogent solution presented, probably in a PEP.

PEP 420 is not well thought out, especially for tools needing to deduce package structure from the file system

Perhaps. The PEP focuses on the import system and not build/packaging systems. The word "discover" appears twice and only in reference to the import system. I wonder how other languages handle this condition - how does Java or Perl (the two languages referenced in the PEPs) deal with package discovery during the build process? Does that provide any inspiration? I know nothing about Perl, but my experience with Java makes me think they're just very explicit about it, not dissimilar to the proposal in #915, or even more explicit without any discovery, such as setup(packages['myns.foo']).

debhelper for Python distributions actually strips the init.py for namespace packages that are declared as such with namespace_packages in the setup call (wanting to encourage PEP 420 I guess, and avoid file conflicts)

Actually, this behavior is baked into setuptools and pip as well. In setuptools, it happens with setup.py install --single-version-externally-managed, which is probably what the deb helper is invoking and what pip invokes. In addition to stripping the __init__.py, it also adds a -nspkg.pth file which provides backward compatibility in older Pythons. In 28.3, Setuptools tried relying entirely on PEP 420 for Python 3.3+, but in version 31, this change was reverted as the -nspkg.pth is necessary to override PEP-420 behavior for namespace packages which have the __init__.py for compatibility reasons.

The result was that developers working on other distributions of ours, with packages in that same namespace package, who set up their PYTHONPATH to point at the distribution they were hacking on, got the dev tool broken since Python couldn't see its PEP 420 package

The release of Setuptools 31 addresses this use case through the use of setup.py develop (invoked by pip install -e), which should be preferred over PYTHONPATH hacks.
And I should point out that if you want to support Python 3.3+ only in your namespace package and rely on PEP 420, that is possible today - you just can't use find_packages but must specify your packages explicitly in setup.py.

__nspkg__ would only be present in the package dir in the distribution, not installed when doing setup.py install, so just a marker for the purpose of getting find_packages working

Yes, though I hadn't thought through it too deeply. The marker would have to be included in sdists. To be conservative, it would be nice if it were not installed to the environment, to prevent its abuse for other purposes.

@estan I modified test_pypi.py a bit to go off installing packages with the modified setuptools. Luckily all the packages install ok with (pip install packagename). However the failure with this plan is that the packages already have the correct files in there. I can't use the whl or tar.gz either, since they both have only the required files in there. So I need to go and find the repos for them in order to see if it works. I started to look at the 'home_page' attribute in the json to detect the github/bitbucket repo. (there's 88 of 400 packages with the required git repo in the home_page). So this will let me download quite a few packages. I'll wait overnight for the repos to finish downloading and continue tomorrow.

@jaraco Thanks for your patient response, and the pointer to PEP 402 (in addition to PEP 420).  The current state-of-play clearly presents a thorny problem, and I understand that making find_packages() work most usefully with the existing import design.  But in addition, the explorations themselves provide important insights on the way to understanding the issues with PEP 420.

@illume: What I did to get that list was just downloaded the first two pages of http://pypi-ranking.info/alltime , and then using pip download -d somedir --no-binary :all: --no-deps <package> to download the source dist for each of them, extract them, and then grep for find_packages in their setup.py to determine if they were using it.
My plan was then simply to create two virtualenvs, one with original setuptools and one with a patched one, and run something like from setuptools import print(find_packages; find_packages(...)) from each of the two virtualenvs against each package, and diff the two lists.

Regarding Java and Go: they're very prescriptive about how developers lay
out their source directories. People put up with that because they're
solely used as application development languages, and not for ad hoc
scripting.
Regarding Perl: if I recall correctly, the package membership declaration
has to appear in the module source file as well as being part of the
directory layout
I think for setuptools, it would be fine to have a "source only" marker
file that was used to lay down a directory on target systems. I'd suggest
"nsdir" as the name, though - you're requesting that the build system
add that directory to the namespace package, not defining the namespace
package directly.

@ncoghlan: Good point about the name (__nsdir__ instead of __nspkg__).
I don't have much more to offer in this discussion I think. Personally I'd be fine with either of these approaches:

An __nsdir__ marker file to guide the search.
A namespace_packages parameter à la my PR to guide the search.
A more liberal search (@illume's suggestion) that consider a directory with any .py files below it an implicit namespace package, with a good set of default excludes, where the user have to use include/exclude to guide the search if the default excludes are not enough.

@estan thanks for the download command. I tried it and had problems, since it tried to call setup.py egg_info on the packages? It's ok though, because I could get the source download link easily from pypi. Unfortunately using the source may be misleading since, find_packages itself determines what goes inside the source file when sdist makes it. So I had to download the git repos as well for a more accurate test. It is good however to use the pypi source downloads to find if find_packages is used, and which arguments are being used.
Anyway, after trying all that now I have about 80 git repos and 186 source packages.
29 out of the 186 packages with source downloaded off pypi out of 400 attempted to download use find_packages with arguments. 19 don't use arguments.
$ find . -name setup.py -depth 2 -exec grep -H find_packages( {} ; | grep -v find_packages() | wc -l
29
$ find . -name setup.py -depth 2 -exec grep -H find_packages() {} ; | wc -l
19
$ find . -name setup.py -depth 2 -exec ls {} ; | wc -l
186
docs and tests are quite common folders. conf.py can be used to filter out docs(a common sphinx folder) or docs/*.rst, and tests/test_*.py seems to be a good filter for the tests dir.
These are what people are using in their exclude= param. ez_setup, docs, examples, tests, examples*, test*, tests.*, benchmarks, contrib, packages, lib.
I will come back with another report after more investigation. Cheers!

Here is a gist that you can include in your setup.py and resolve PEP420 namespace packages and standard packages (those with an init.py)
https://gist.github.com/silkentrance/2710c559f07560d87d33cb6698dfe7a1
It does not find namespace packages that use the old scheme, though. I have code here, that does that, but it is unreliable.

I've opened #1140 to resolve this.

@jaraco since you rejected #1140, the only way to do this is via setup.cfg or settings packages to find:. Agreed?

@agronholm I apologize. I didn't mean to be rejecting #1140 outright. It did highlight two alternate paths, vendoring OrderedDict or dropping support for Python 2.6, such that the namespace_packages declaration can be re-used. But now that I think about it more, we probably don't want to re-use that declaration, because it does more than simply declare namespace packages. It also signals the behavior that creates -nspkg.pth files and other namespace package management, which we may not want conflated with package discovery.
As I re-read this issue and consider the two PRs, I guess there is a consensus that we should add this interface. We should probably base the solution on #943 as I promised to do there.

the only way to do this is via setup.cfg or settings packages to find:

I'm not sure I follow. And the declarative config, including find: didn't exist at the time. Does find: already support declaration of namespace packages?

I'm not sure I follow. And the declarative config, including find: didn't exist at the time. Does find: already support declaration of namespace packages?

No, but unlike find_packages(), it could be made to reuse the existing declaration. That declaration in turn can be made not to create nspkg.pth files on Python 3, where PEP 420 namespace packages already work. Would there be a problem with this?

Note: if the only Python 2.6 you care about is the one in RHEL 6, that actually provides collections.OrderedDict (other things in RHEL needed it, so it got backported a few years back).

I don't like #943. It would consider my tests directory a namespace package because it contains Python modules. Ditto for docs.

Yes, I agree. #1140 is better. I'll revive it.

unlike find_packages(), find: could be made to reuse the existing declaration

That's in intriguing idea. If we could deprecate find_packages altogether or at least make find: the preferred mechanism for package discovery, perhaps that would be the best way. I do really want to avoid having two mechanisms, each of which have unique features and detriments.

So the new consensus would be to deprecate find_packages() in favor of find: + namespace_packages. The latter would be changed to not require a placeholder __init__.py in the namespace package directory. I will look into making a PR with these changes.

Is there any progress on this topic?
PEP 420 was approved by Guido on May 24, 2012
This issue was created on Nov 5, 2013
And the only output is a documentation change (roughly 5 years later) that find_package does not support PEP 420!?
EOL of Python 2.7 is January 1st, 2020.
Python 3.3 which introduced this feature the first time has already reached EOL
So basically everyone should switch to a modern Python version, which support PEP 420, in the next few month, but there is no useful way to use it within setuptools?

So basically everyone should switch to a modern Python version, which support PEP 420, in the next few month, but there is no useful way to use it within setuptools?

What is preventing you from manually specifying the packages? That's what I do.

What is preventing you from manually specifying the packages? That's what I do.

Yes, of course, because there is no other option.
But what's the meaning of find_packages if it does not work with modern Python?

But what's the meaning of find_packages if it does not work with modern Python?

I'll remind that PEP 420 namespaces is a feature not needed by most people so that statement seems a bit unfair. That said, this will probably require a PR before it goes anywhere.

This issue was created on Nov 5, 2013
"You're welcome".
And 50% of my rationale for raising this issue was that PEP420 seemed to break what people relied upon to indicate a package, further weakening Python's chronic lack of explicit definition or record of programmers' intent as to what constitutes a complete "thing", a la "project" in other development environments. (And obliquely, agronholm's comment "I'll remind that PEP 420 namespaces is a feature not needed by most people" somewhat reinforces that observation.)  For all that I like about Python, this is a problem area in my view, and continues to be.

@agronholm I believe that most of the functionality is already there. Yet, PEP420PackageFinder.find is not being exported yet and might be done so via find_packages_ns.
I am currently working on a PR for this.

Has anyone tried using the interface currently presented by Setuptools, namely:
setup(
    packages=setuptools.PEP420PackageFinder.find(),
)

Most of my projects have migrated or are migrating their metadata and options to setup.cfg. How do I use that from setup.cfg?

Has anyone tried using the interface currently presented by Setuptools, namely:
setup(
    packages=setuptools.PEP420PackageFinder.find(),
)

I tried it now and with using the exclude parameter it would work for me.
But what do you mean with currently presented? Will you remove this interface again? Should this be an official interface everyone could/should use? At the moment one of PyCharms inspection reports a warning

@jaroco feel free to contribute to the PR

@agronholm the official documentation https://docs.python.org/3/distutils/configfile.html lacks a lot, do you have more information?

the official documentation https://docs.python.org/3/distutils/configfile.html lacks a lot, do you have more information?

Regarding?

@pganssle How do I use find_packages_ns from setup.cfg? It's not obvious from your commit.

@agronholm It was @silkentrance that did the implementation, I just merged it. Good point, though, I don't believe this is supported. I will create a new issue.


WIP: Sendable annotations

This is currently a net-neutral in terms of warnings. Previously we got two through not having sendable annotations at all, now we get two for mutable properties in types annotated as sendable.

https://github.com/apple/swift-tools-support-core/pull/367
@swift-ci please smoke test

Hm, interesting, looks like some of the sendable diagnostics are somehow errors in the macOS job? Are we doing something special in bootstrap?
/Users/ec2-user/jenkins/workspace/swift-package-manager-with-xcode-self-hosted-PR-osx/branch-main/swiftpm/Sources/PackageModel/PackageModel.swift:62:16: error: stored property '_targets' of 'Sendable'-conforming class 'Package' is mutable
    public var targets: [Target]
               ^

@DougGregor ?

Oh I see, we just have an error in macOS self-hosted, so it's an issue of different behavior in Swift 5.6

Doesn't seem as if this is actually solvable. I see two major issues here:

some warnings are errors in older swift, e.g. ```/Users/neonacho/Projects/public/swift-package-manager/Sources/PackageModel/PackageModel.swift:62:16: error: stored property '_targets' of 'Sendable'-conforming class 'Package' is mutable
public var targets: [Target]
^

- there are errors about e.g. `URL` which were supposedly fixed in newer SDKs

So I think this needs to wait until we have upgraded to a newer base version of Swift (at least 5.7).

This has been obsoleted at this point.


Cannot stream depth, color, infrared in one window using pyrealsense2

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
D435

Firmware Version
<IP_ADDRESS>

Operating System & Version
Ubuntu 20.4.3

Kernel Version (Linux Only)
5.11.0-34-generic

Platform
PC

SDK Version
2.49.0

Language
python

Segment
others

Issue Description
I'm trying to gather the depth, color, infrared data using D435 but when I put everything together with hconcat, vconcat or vstack, hstack generates an index error in the arrays. have any idea how i could do it, this is my code.
import pyrealsense2 as rs
import numpy as np
import cv2

# Configuracion de las camaras para el stream
pipeline = rs.pipeline()
config = rs.config()
#configuracion de la camara de progundidad
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
#configuracion de la camara RGB
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 15)
#configuracion de los dos infrarojos
config.enable_stream(rs.stream.infrared, 640, 480, rs.format.y8, 30)

align_to = rs.stream.color
align = rs.align(align_to)

# Inicia el stream con las configuraciones hechas
pipeline.start(config)

# Ingnora el primer segundo para que se inicialize la camara
for i in range(30):
    frames = pipeline.wait_for_frames()

try:
    while True:

        # Solicita los primeros frames de las camaras
        frames = pipeline.wait_for_frames()
        #frames de profundidad
        aligned_frames = align.process(frames)
        depth_frame = aligned_frames.get_depth_frame()
        #frames RGB
        color_frame = frames.get_color_frame()
        #frames IR
        infrared_frame = frames.first(rs.stream.infrared)
        if not depth_frame or not color_frame:
            continue

        # Convierte las imagenes en un array numpy
        Infra_image = np.asanyarray(infrared_frame.get_data())
        depth_image = np.asanyarray(depth_frame.get_data())
        color_image = np.asanyarray(color_frame.get_data())

        # Se aplica un mapa de color a la imagen de profundida las imagenes deben ser convertidas de 8bits a pixeles
        depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET)

        # Aplila las camaras en una sola ventana
        images1 = cv2.hconcat([color_image, depth_colormap])
        images2 = cv2.hconcat([color_image, Infra_image])
        final = cv2.vconcat([images1, images2])
        # Muestra las camaras en una ventana
        cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)
        cv2.imshow('RealSense', final)
        #cv2.imshow('Imagen RGB', color_image)
        #cv2.imshow('Imagen de Profundidad', depth_colormap)
        #cv2.imshow('IR image', Infra_image)

         # Al presionar ESC termina el programa
        c = cv2.waitKey(1) % 0x100
        if c == 27:
            break
finally:
    #Detiene el stream
    pipeline.stop()
    cv2.destroyAllWindows()

This is the error:
Traceback (most recent call last):
  File "prueba.py", line 50, in <module>
    images2 = cv2.hconcat([color_image, Infra_image])
cv2.error: OpenCV(4.5.1) /tmp/pip-req-build-jr1ur_cf/opencv/modules/core/src/matrix_operations.cpp:66: error: (-215:Assertion failed) src[i].dims <= 2 && src[i].rows == src[0].rows && src[i].type() == src[0].type() in function 'hconcat'

Hi @Chepo2393  The case https://github.com/IntelRealSense/librealsense/issues/5093 demonstrates the approach that a RealSense user took to aligning depth, infrared and color after advice from a RealSense team member.

I already tried to align the flows but I realized that the error is when I do the combination of IR with RGB or depth
align_to = rs.stream.infrared
align = rs.align(align_to)
...
        frames = pipeline.wait_for_frames()
        aligned_frames = align.process(frames)
        depth_frame = aligned_frames.get_depth_frame()
        color_frame = aligned_frames.get_color_frame()
        infrared_frame = frames.first(rs.stream.infrared)
...
images2 = cv2.hconcat([color_image, Infra_image])

An alternative approach that another RealSense user took in https://github.com/IntelRealSense/librealsense/issues/5963#issuecomment-638403524 was to align depth to RGB and then align left infrared to depth, so that they ended up with the left infrared aligned to depth and color.

Hi @Chepo2393  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.

HI @MartyG-RealSense, I would like to discuss this issue again. The realsense-viewer software uses the infrared data to help the depth sensors calculating the depth. One way to see that is to enable the stereo in the software and if you put your finger in front of the infrared camera you can see that we lose details and the capability of calculating the depth gets worse as it is visible in the stream.
I would like to use this the infrared camera to help the depth cameras too, but in python. How can I do it? Thanks!

Hi @pfcouto  On RealSense 400 Series stereo cameras there are a pair of left and right infrared sensors (hence the name 'stereo') that are used to construct the depth image.  Putting a finger over either of the infrared sensors will cause the depth image to be obscured.
You do not actually need to enable the infrared streams to obtain a depth image though.  The depth image is not constructed by the Infrared and Infrared 2 streams in the Viewer, but from raw left and right IR frames inside the camera hardware at the point of capture.  The assembled depth frame is then sent along the USB cable to the computer.  The depth frame can therefore be obtained even if both Infrared streams are not enabled.

Most RealSense models (not all of them) are equipped with a separate projector component that casts a semi-random infrared dot pattern onto the surfaces of objects in the scene.  The camera can use these dots as a 'texture source' to analyze for depth detail.  Increasing the Laser Power option's value makes the dots more visible to the camera and may increase the amount of depth detail on the image, whilst reducing Laser Power reduces the visibility of the dots and can cause the amount of detail on the depth image to be reduced.
400 Series cameras can also use ambient light in a scene to aid analysis of surfaces instead of using the projector's dot pattern if the scene is well lit.

Yes. That's what I am refering to. I am using a d400 series camera - d435i and it has the IR Projector, can't I use this feature in python code to improve the depth quality?
Also, using this picture as an example, with the following line of code (depth_frame.get_distance(109, 280)) what am I calculating, Deth or Range? In my case if I have two objects on the same plan, but one is right in the center of the stream and the other one is further way to the side the distance calculated to both will be the same or really close, I believe this is because I am getting the depth and not the range (real distance to object, not to the plane it is on) as I would like.
How can Iget the Range instead of the Depth? Thanks a lot!!

Yes, you can enable the projector with Python code by setting the SDK instruction rs.option.emitter_enabled to '1'.    Below is an example of such code.
import pyrealsense2 as rs
pipeline = rs.pipeline()
config = rs.config()
pipeline_profile = pipeline.start(config)
device = pipeline_profile.get_device()
depth_sensor = device.query_sensors()[0]
if depth_sensor.supports(rs.option.emitter_enabled):
depth_sensor.set_option(rs.option.emitter_enabled, 1)

The above image that you quoted is from page 17 of the current edition of the data sheet document for the 400 Series.
https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet
You are correct, the depth value is representative of the depth and not the range.  Text accompanying the image in the data sheet states: "The depth pixel value is a measurement from the parallel plane of the imagers and not the absolute range as illustrated".
The depth value is equivalent to measured real-world distance in meters, with a small margin of error compared to a physical real-world tape measure from the camera lens to the observed object.
If you generate a pointcloud then you can obtain real-world 3D XYZ world coordinates from its vertices, like in the SDK's Python example program opencv_pointcloud_viewer.py
https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/opencv_pointcloud_viewer.py

Ok, I will try it out. Currently I am using this piece of code. This does not use the IR Project automatically right? If I use a code similar to the one you exemplified I should get a better result, or not?

The image below represents my problem. I have two water bottles, they are in the same plane (or pretty close at least) so the depth is the same in both. However what I want is to get the range to both. In reality to the small water bottle it is accurate - 1.78m (or close) - as it is in the center of the capture, but that is not the case for the bigger one, in reality the range is 1,97m. How can I calculate this range values? As you can see I am also calculating the angles - horizontal and vertical - using the a fov, but I don't know the correct Fov to use, so maybe I can try to calculate the range with this? Right now I am using the Fov of the Depth_VGA but should I be using another one? Depth_VGA, Depth_HD or Color_Camera? Thanks a lot!!!

The RealSense Viewer applies a range of configuration presets by default when launched, whilst for self-created scripts you need to manually program those settings into your scripting in order to replicate the images produced by the Viewer.  You should enable the IR Emitter in your script with the provided code in order to project a depth-enhancing dot pattern onto the scene
The official RealSense store page and the official information page for the D435i both quote the Depth HD FOV of 87 x 58 degrees so I would recommend using the Depth HD FOV.
On the D435i camera model the color FOV is smaller than the depth FOV, so you may get more accurate results if you align depth to color so that the depth FOV is automatically resized to match the color FOV size.  Because the depth and color sensors are at different physical positions on the front of the camera, the color view will be horizontally offset from the depth sensor's view unless the two streams are aligned.

Isn't the alignment done by the line frames = self.align.process(frames) shown in the first print? So you think that I should use the Depth_HD FOV instead? On the process to align, am I aligning to de Color or Depth? If it is to the color maybe using the color FOV would be wiser.
What about the range? If there is no way to get the range using the SDK can you provide a way to calculate it, like using the angles of the point using the center point as measure, because the central point is the normal of the camera or some other way? Thanks.

Could you provide a link to your previous answer that contains the line frames = self.align.process(frames) please?
If you have that line in your code then you are already using alignment in your project, yes.
Yes, I would recommend using Depth FOV HD (87 x 58 degrees).
Most people perform depth to color alignment.  It is very rare that color to depth alignment is used.
The main difference between them is that on D435i (where the color FOV is smaller than the depth FOV), the outer edges of the aligned image may be cut off.  This is because the smaller color FOV means that there is not color coordinates at the edges of the image to match up to depth coordinates.
If color to depth alignment is used on D435i then the smaller color FOV means that the color image will be stretched to the size of the larger depth FOV, so the aligned image will fill the screen and not have information cut off at the edges.

It sounds as though the information at https://github.com/IntelRealSense/librealsense/issues/9297 about finding an object's height (distance from ground) using the angle might be a helpful reference for your range question.

OK, so in my code I have self.align = rs.align(rs.stream.color) that means I am aligning the color to depth right? If that's the case then I should be using the Depth FOV and not the Color FOV. Or is ir the opposite and I am aligning the depth to the color and then I should use the Color FOV? Sorry for all the trouble and thank you!

It's no trouble at all  :)
rs.stream.color in the brackets means that it is aligning depth to color, as whatever is in the brackets is the target stream that is being aligned to.  To align color to depth, you change the brackets to rs.stream.depth
Whilst the majority of RealSense users align depth to color, which one you use will depend on which works best for your particular project.

OK, then the correct FOV to use should be the Color one, correct? I might be getting somewhere with the range using the angles so overcoming this problem of the FOVs would be awesome. Thanks!

rs.stream.color is the FOV that most people align to.  Try both though and see which one provides the most accurate results, since it is easy to test both modes just by changing the bracket contents.
I will be pleased to continue this discussion with you tomorrow if you have further questions.  Good luck!


Connect-MicrosoftTeams not working in GCC High when using -AadAccessToken

In the process of building out a SPO Site/Teams provisioning solution for a client, I have stumbled upon a but with the Connect-MicrosoftTeams cmdlet and am hoping to get some assistance.
For context, my solution uses an AAD app (running with app-only permissions) to extend SharePoint Site Designs through an Automation Runbook in Azure (running PowerShell). However, the issue reported below was run locally to limit variables here.
Issue
When connecting to Connect-MicrosoftTeams for a GCC High tenant, the following property must be supplied: -TeamsEnvironmentName TeamsGCCH. This works fine when authenticating via a prompt with a username and password. However, this fails when authenticating by supplying a -AadAccessToken parameter. See error below:

The issue looks to be unique to the combination of -TeamsEnvironmentName and -AadAccessToken as if I omit either, the connection will work (although won't suffice for my solution).
Working by omitting the -TeamsEnvironmentName parameter:

Working by omitting the -AadAccessToken parameter:

Please advise! Based on my testing, Connect-MicrosoftTeams simply doesn't work when needing to both connect to a GCC High tenant and needing to pass parameters to authenticate via a token (app-only) instead of requiring user-input.

@officedocsbot assign @yogkumgit

Hello @patrickabel How are you?
Yes, they are in different parameter sets so currently you won't be able to use them at the same time.
I know this information could be useful as feedback for @tameemansari

@get-itips  Thank you very much for the contribution and sharing this explanation. @patrickabel
Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


[Feature] Add ecrecover

Component
consensus, eips, genesis
Describe the feature you would like
alloy itself exposes recover_signer yet not a 1:1 implementation of ecrecover.
ethers-rs did expose a recover function, internally calling the ecdsa crate.
ecdsa doesn't explicitly implement ecrecover, as defined in Ethereum. Please note their policy on low/high s values: https://github.com/RustCrypto/elliptic-curves/issues/991
This did break an EVM dependent on it (which bubbled up into foundry IIRC).
While I could implement ecrecover myself based on ecdsa, I do not want to have to review the ecdsa crate for their current policies/impacted functions. I'd like a function which just works and is guaranteed to work in this context. While I do not believe the ecdsa crate's policies are expected to change, I'll note they require additional code around their recovery in order for their recovery to be used as ecrecover (justifying the definition of a dedicated ecrecover function regardless).
Additional context
No response

This is already implemented in alloy_primitives::Signature::recover*, see also this test for recovering a non-normalized signature.

Apologies. I looked through the docs and couldn't find it. I don't believe they're generating properly for Signature.

Yes you're right, could you open an issue in alloy-rs/core?


Line numbers in clone refactor output are sometimes incorrect

Additional insight: this is caused by the type 2R transformation. This component should be rewritten anyway, so that'll probably be solved when that is done.


README-dev.md: add tips when updating the bindings

I think closing https://github.com/o1-labs/snarkyjs-bindings/pull/83 is ok to do. I thought duplicating the info would be useful as the dev might have a look at the different readmes, but if they want to run the tests, it is the readme from the repo they will have a look at.


Behaviour when trying to access the payload after responding

Description:
$title.
Steps to reproduce:
import ballerina/http;
import ballerina/io;

service foo on new http:Listener(8080) {

    @http:ResourceConfig {
        methods: ["POST"]
    }
    resource function bar(http:Caller caller, http:Request req) {
        var result = caller->respond("Accepted!");
        io:println("BAR");
        io:println(req.getJsonPayload());
        io:println("BAR2");
    }
}

When invoking the service, execution seems to hang at req.getJsonPayload() eventually resulting in a timeout error.
$ curl -v http://localhost:8080/foo/bar -d '{ "name": "Anne", "age": 25 }' -H "Content-Type:application/json"

$ ballerina run foo.bal
[ballerina/http] started HTTP/WS listener <IP_ADDRESS>:8080
BAR
error {ballerina/http}GenericClientError message=Error occurred while retrieving the json payload from the request cause=error {ballerina/mime}ParsingEntityBodyFailed message=Error occurred while extracting json data from entity: No entity was added to the queue before the timeout
BAR2

As pointed out by @arukshani, this works if we've accessed the payload prior to responding.
Is this the correct behaviour?

don't we allow accessing the payload for the first time after responding?
if we don't do we still need the timeout?

Affected Versions:
$ ballerina -v
Ballerina 1.0.0
Language specification 2019R3

The reason for releasing content at the final state of response path is to avoid potential memory leak ( ballerina-platform/ballerina-lang#12815 ) which can occur when the request is not dispatched to a particular resource.
One thing we can do is moving that content release to all possible inbound request failure paths.

And do a load test by monitoring memory behaviour in following cases
1. Successful return value from resource flow
2. checkpanic from resource flow
3. Dispatching/data binding response flow
4. OPTIONS call response flow

Following are the load test profiling results for above scenarios after proposed changed. (Load test concurrency 50)

Successful return value from resource flow

checkpanic from resource flow

Dispatching/data binding response flow

OPTIONS call response flow

Reopen due to https://github.com/ballerina-platform/ballerina-standard-library/issues/1731


Error after upgrade to 0.6.3

Hi, after upgrading to 0.6.3 when I try to load a YAML schema I have receive the following error:
[2021-04-29 00:22:21,471] {taskinstance.py:1396} ERROR - 'checks'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1086, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/usr/local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1260, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/usr/local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1300, in _execute_task
    result = task_copy.execute(context=context)
  File "/usr/local/airflow/dags/common/operators/db/sql_to_s3_operator.py", line 60, in execute
    df = self.query()
  File "/usr/local/airflow/dags/common/operators/db/mssql_table_to_s3_operator.py", line 95, in query
    schema = pa.DataFrameSchema.from_yaml(self.db_table_schema)
  File "/usr/local/airflow/.local/lib/python3.8/site-packages/pandera/schemas.py", line 1195, in from_yaml
    return pandera.io.from_yaml(yaml_schema)
  File "/usr/local/airflow/.local/lib/python3.8/site-packages/pandera/io.py", line 242, in from_yaml
    return _deserialize_schema(serialized_schema)
  File "/usr/local/airflow/.local/lib/python3.8/site-packages/pandera/io.py", line 206, in _deserialize_schema
    if serialized_schema["checks"] is not None:
KeyError: 'checks'

My YAML schema has some null checks like this
ID:
pandas_dtype: int64
nullable: false
checks: null
How I should specify columns without checks?

@jeffzi  thanks for your answer. It didn't find any documentation about the changes in the 0.6.3 schema. With your answer is clear to me what I need to do.
Regards!

let's re-open this so that @jeffzi can reference it in the PR

fixed by #479 thank to @antonl


The continue plugin has been initializing......

Before submitting your bug report

[X] I believe this is a bug. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that reports the same bug
[X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info
- OS: macos 14.4.1
- Continue: 0.8.27
- IDE: vs code 1.90.1
- Ollama 0.1.44

I use VS Code + Ubuntu server, remote ssh mode of development.
The project is huge, according to incomplete statistics there are 30579 directories, 184726 files.

  "models": [
    {
      "title": "ollama-codeqwen:7b-chat-v1.5-q8_0",
      "provider": "ollama",
      "model": "codeqwen:7b-chat-v1.5-q8_0",
      "contextLength": 8192,
      "temperature": 0
    }]

  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text",
    "apiBase": "http://localhost:11434"
  }

Description
The continue plugin has been initializing... No files in any of the project directories are actually indexed, so features like @folder and @files are not available.
The official document method similar to exclude https://docs.continue.dev/walkthroughs/codebase-embeddings#customizing-which-files-are-indexed mentioned way, there is no include method.
I'm trying to ~/continue/index/index.splite looked at the indexfile, some files have not been indexed, regardless of the file size.

To reproduce
No response
Log output
workbench.desktop.main.js:sourcemap:152 [Extension Host] EntryNotFound (FileSystemError): Unable to resolve nonexistent file 'vscode-remote://ssh-remote/home/ubuntu/gerde/it17.main/.prompts'
	at y.e (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:152:6797)
	at Object.readDirectory (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:152:4081)
	at async traverseDirectory (/Users/chr/.vscode/extensions/continue/out/extension.js:369145:17)
	at async _VsCodeIdeUtils.getDirectoryContents (/Users/chr/.vscode/extensions/continue/out/extension.js:369442:26)
	at async VsCodeIde.listWorkspaceContents (/Users/chr/.vscode/extensions/continue/out/extension.js:369825:18)
	at async getPromptFiles (/Users/chr/.vscode/extensions/continue/out/extension.js:344431:19)
	at async Promise.all (index 0)
	at async serializedToIntermediateConfig (/Users/chr/.vscode/extensions/continue/out/extension.js:344579:24)
	at async loadFullConfigNode (/Users/chr/.vscode/extensions/continue/out/extension.js:344845:22)
	at async ConfigHandler.loadConfig (/Users/chr/.vscode/extensions/continue/out/extension.js:344969:28)
	at async ConfigHandler.getSerializedConfig (/Users/chr/.vscode/extensions/continue/out/extension.js:344952:30)
	at async /Users/chr/.vscode/extensions/continue/out/extension.js:366811:18
	at async a.value (/Users/chr/.vscode/extensions/continue/out/extension.js:367260:32)
w @ workbench.desktop.main.js:sourcemap:152
workbench.desktop.main.js:sourcemap:152 [Extension Host] stack trace: EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, stat '/home/ubuntu/gerde/it17.main/na/res/CN/BE'
	at y.e (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:152:6797)
	at Object.stat (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:152:3892)
	at async /Users/chr/.vscode/extensions/continue/out/extension.js:369790:25
	at async Promise.all (index 16549)
	at async VsCodeIde.getStats (/Users/chr/.vscode/extensions/continue/out/extension.js:369788:9)
	at async CodebaseIndexer.refresh (/Users/chr/.vscode/extensions/continue/out/extension.js:345105:25)
	at async _VsCodeExtension.refreshCodebaseIndex (/Users/chr/.vscode/extensions/continue/out/extension.js:370658:26)

Add: ssh way development, file path is valid, the file exists.

+1

+1

+1

hello, which version are you using?