Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion differences.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,15 @@ TalkBack ignored the order for a row whose checkbox or switch has no text of its

### "Backtalk off" at the right volume

When you turn Backtalk off, it says "Backtalk off" at the accessibility volume, using your device's real volume levels. TalkBack only approximated that volume.
When you turn Backtalk off, it says "Backtalk off" at the accessibility volume, using your device's real volume levels. This includes Samsung phones, and stays right when you connect or disconnect headphones while Backtalk is on. On Bluetooth headphones that control their own volume, it can be one volume step louder or quieter, about 3 dB, because those headphones can set the media volume between steps and Android doesn't tell apps where. TalkBack only approximated that volume.

### Volume said as it changes

When you press a volume key, Backtalk says the new volume right away and stops what it was saying. During continuous reading it waits for the current item to finish instead, so reading goes on. A newer level replaces one that hasn't finished. The first change names the volume, such as "Music volume set to 50 percent", and changes to the same volume less than 2 seconds apart say only the percentage. If you hold the key down, Backtalk says the first level, stays quiet while the volume keeps changing, and says the level you end at when you let go. When media is muted or unmuted, Backtalk says so the same way. TalkBack waited until the keys had been still for 2 seconds, and after the first press it waited 4 seconds. Backtalk also says the media volume while media plays, where TalkBack said it only when nothing was playing.

Each output, such as the phone speaker or a pair of headphones, keeps its own media, call, notification and accessibility volumes, and Android switches to them when audio moves there. Backtalk doesn't say a change of one of these volumes that comes with a new output unless a volume key made it, so connecting or disconnecting headphones stays quiet. Headphones can report the same level again a moment later, and Backtalk doesn't say one of these levels that hasn't changed unless a volume key reported it. TalkBack said the level headphones set. If the new output's level is the one you already had, nothing changes, so the first press on the headphones' own controls after that isn't said.

When you change the volume with a slider, Backtalk says only the slider's value. To stop hearing the volume, including mute, turn off **Verbosity** > **Speak volume changes**.

### Table column headers

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,19 @@ import android.media.AudioAttributes
import android.media.AudioDeviceInfo
import android.media.AudioManager
import android.os.Build
import com.google.android.accessibility.talkback.audio.outputsFor
import kotlin.math.pow

/**
* Works out how loud to speak "Backtalk off".
*
* Android shuts the accessibility audio stream down before Backtalk speaks that last announcement,
* so it plays at the media volume. To keep it at the accessibility volume, the speech is made
* quieter by the difference between the two. Android 9 and later report each volume step's real
* loudness in decibels for the current output, which is exact. Before that, the difference is
* guessed from where the two volume sliders are, which lowers the speech too little at the
* quietest accessibility volumes, where Android's volume steps are furthest apart.
* Android turns the accessibility volume up to the media volume's step before Backtalk speaks that
* last announcement. To keep it at the accessibility volume, the speech is made quieter by the
* difference between the two steps. Android 9 and later report each step's real loudness for the
* current output, which is exact. The accessibility stream's own loudness is used for both,
* because the media stream's can differ from it. Before Android 9, the difference is guessed from
* where the two volume sliders are, which lowers the speech too little at the quietest
* accessibility volumes, where Android's volume steps are furthest apart.
*/
object FinalAnnouncementVolume {
/**
Expand Down Expand Up @@ -59,14 +61,51 @@ object FinalAnnouncementVolume {
return 0f
}
val gain =
decibelGain(audioManager, accessibilityIndex, musicIndex)
reportedGain(audioManager, accessibilityIndex, accessibilityMaxIndex, musicIndex, musicMaxIndex)
?: sliderGain(
accessibilityIndex.toFloat() / accessibilityMaxIndex,
musicIndex.toFloat() / musicMaxIndex,
)
return speechVolume * gain
}

/**
* The accessibility step that Android sets for media step [musicIndex], the way it matches the
* steps of two streams: in tenths of a step, from each stream's lowest step to its highest, then
* rounded. Bluetooth headphones can set the media volume between steps, which apps can't see,
* so there this can be a step off.
*/
@JvmStatic
fun accessibilityStepAtMediaVolume(
musicIndex: Int,
musicMinIndex: Int,
musicMaxIndex: Int,
accessibilityMinIndex: Int,
accessibilityMaxIndex: Int,
): Int {
val musicRange = (musicMaxIndex - musicMinIndex) * 10
val accessibilityRange = (accessibilityMaxIndex - accessibilityMinIndex) * 10
val tenths =
accessibilityMinIndex * 10 +
((musicIndex - musicMinIndex) * 10 * accessibilityRange + musicRange / 2) / musicRange
return (tenths + 5) / 10
}

/**
* The gain that turns a sound at the media step into one at the accessibility step, from 0 to 1,
* from the loudness Android reports for each. That is usually in decibels, which are at most
* 0 at full volume, but Samsung phones report a fraction of full loudness instead, from 0 to 1.
* Their fractions are above 0 at every step but the lowest, which is never asked for, so two
* values above 0 are fractions.
*/
@JvmStatic
fun gainFromReportedVolumes(accessibility: Float, music: Float): Float {
if (accessibility > 0f && music > 0f) {
return (accessibility / music).coerceIn(0f, 1f)
}
return gainFromDecibels(accessibility, music)
}

/** The gain that turns a sound at [musicDb] into one at [accessibilityDb], from 0 to 1. */
@JvmStatic
fun gainFromDecibels(accessibilityDb: Float, musicDb: Float): Float {
Expand All @@ -88,23 +127,37 @@ object FinalAnnouncementVolume {
return 10f.pow((accessibilityFraction - musicFraction) / 0.4f)
}

private fun decibelGain(
/**
* The gain from the accessibility stream's loudness at [accessibilityIndex] and at the step
* Android sets it to for media step [musicIndex].
*/
private fun reportedGain(
audioManager: AudioManager,
accessibilityIndex: Int,
accessibilityMaxIndex: Int,
musicIndex: Int,
musicMaxIndex: Int,
): Float? {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.P) {
return null
}
val mediaStep =
accessibilityStepAtMediaVolume(
musicIndex,
audioManager.getStreamMinVolume(AudioManager.STREAM_MUSIC),
musicMaxIndex,
audioManager.getStreamMinVolume(AudioManager.STREAM_ACCESSIBILITY),
accessibilityMaxIndex,
)
val deviceType = outputDeviceType(audioManager)
return try {
gainFromDecibels(
gainFromReportedVolumes(
audioManager.getStreamVolumeDb(
AudioManager.STREAM_ACCESSIBILITY,
accessibilityIndex,
deviceType,
),
audioManager.getStreamVolumeDb(AudioManager.STREAM_MUSIC, musicIndex, deviceType),
audioManager.getStreamVolumeDb(AudioManager.STREAM_ACCESSIBILITY, mediaStep, deviceType),
)
} catch (e: IllegalArgumentException) {
null
Expand All @@ -119,7 +172,7 @@ object FinalAnnouncementVolume {
.setUsage(AudioAttributes.USAGE_MEDIA)
.setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
.build()
audioManager.getAudioDevicesForAttributes(attributes).firstOrNull()?.let {
audioManager.outputsFor(attributes)?.firstOrNull()?.let {
return it.type
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1427,6 +1427,12 @@ protected boolean onKeyEventInternal(KeyEvent keyEvent) {
if (adjustSliderWithVolumeKey(keyEvent, triggerModifierPressed)) {
return true;
}
// Backtalk: a held volume key is said once, when it's released. This follows the keys that
// Backtalk routes to its own volume too, so the release is seen even if routing changes.
if (volumeMonitor != null
&& (keyCode == KeyEvent.KEYCODE_VOLUME_DOWN || keyCode == KeyEvent.KEYCODE_VOLUME_UP)) {
volumeMonitor.onVolumeKeyEvent(keyEvent);
}
if (keyAction == KeyEvent.ACTION_DOWN) {
boolean handleVolumeKeyInTalkBack =
isTouchInteracting
Expand Down Expand Up @@ -2578,6 +2584,15 @@ public long getSpeechCacheWindowCounts() {
focuser.setMenuManager(menuManager);

volumeMonitor = new VolumeMonitor(pipeline.getFeedbackReturner(), this, callStateMonitor);
// Backtalk: a focused slider that the user just moved says its own value, and continuous
// reading must not be cut off. A change from elsewhere, such as headphones, is still said.
volumeMonitor.setSliderFocused(
() ->
RecentUserInput.isRecent()
&& Role.getRole(accessibilityFocusMonitor.getSupportedAdjustableNode())
== Role.ROLE_SEEK_CONTROL);
volumeMonitor.setContinuousReading(
() -> pipeline.getActorState().getContinuousRead().isActive());
selectorController =
new SelectorController(
this,
Expand Down Expand Up @@ -3195,6 +3210,7 @@ private void resumeInfrastructure() {
if (volumeMonitor != null) {
ContextCompat.registerReceiver(
this, volumeMonitor, volumeMonitor.getFilter(), RECEIVER_EXPORTED);
volumeMonitor.startAnnouncing();
if (FeatureSupport.hasAccessibilityAudioStream(this)) {
// Cache the initial volume in case that the volume is never changed during runtime.
volumeMonitor.cacheAccessibilityStreamVolume();
Expand Down Expand Up @@ -3347,6 +3363,7 @@ private void suspendInfrastructure() {
if (volumeMonitor != null) {
volumeMonitor.releaseControl();
volumeMonitor.clearVolumeChangedListener();
volumeMonitor.stopAnnouncing();
}

if (headphoneStateMonitor != null) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -308,21 +308,11 @@ class AudioDeviceRouter @JvmOverloads constructor(
/** Whether [device] is Bluetooth and Android plays media on it. */
private fun playsAsMedia(am: AudioManager, device: AudioDeviceInfo): Boolean {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.S) return false
// Where media plays now. The communication route doesn't change it. Android 12 can't say, so
// any connected Bluetooth media output counts.
val media =
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
// Where media plays now. The communication route doesn't change it.
am.getAudioDevicesForAttributes(
AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build()
)
} else {
// Android 12 can't say, so any connected Bluetooth media output counts.
am.getDevices(AudioManager.GET_DEVICES_OUTPUTS).toList()
}
} catch (e: Exception) {
Log.w(TAG, "Couldn't find where media plays", e)
return false
}
am.outputsFor(AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build())
?: return false
return playsAsMedia(endpointOf(device), media.map(::endpointOf))
}

Expand Down Expand Up @@ -398,20 +388,15 @@ class AudioDeviceRouter @JvmOverloads constructor(
*/
private fun accessibilityAudioGoesToTarget(am: AudioManager, targetDevice: AudioDeviceInfo): Boolean {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.TIRAMISU) return true
return try {
val attrs =
AudioAttributes.Builder()
.setUsage(AudioAttributes.USAGE_ASSISTANCE_ACCESSIBILITY)
.setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
.build()
val devices = am.getAudioDevicesForAttributes(attrs)
if (devices.isEmpty()) return true
val onSpeaker = devices.any { it.type == AudioDeviceInfo.TYPE_BUILTIN_SPEAKER }
if (targetDevice.type == AudioDeviceInfo.TYPE_BUILTIN_SPEAKER) onSpeaker else !onSpeaker
} catch (e: Exception) {
Log.w(TAG, "getAudioDevicesForAttributes failed", e)
true
}
val attrs =
AudioAttributes.Builder()
.setUsage(AudioAttributes.USAGE_ASSISTANCE_ACCESSIBILITY)
.setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
.build()
val devices = am.outputsFor(attrs)
if (devices.isNullOrEmpty()) return true
val onSpeaker = devices.any { it.type == AudioDeviceInfo.TYPE_BUILTIN_SPEAKER }
return if (targetDevice.type == AudioDeviceInfo.TYPE_BUILTIN_SPEAKER) onSpeaker else !onSpeaker
}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* Copyright 2026 Backtalk contributors
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/

package com.google.android.accessibility.talkback.audio

import android.media.AudioAttributes
import android.media.AudioDeviceInfo
import android.media.AudioManager
import android.os.Build
import android.util.Log

/**
* The devices audio with [attributes] plays on, or null when Android can't say. Before Android 13
* apps can't ask, so the connected outputs stand in for them.
*/
fun AudioManager.outputsFor(attributes: AudioAttributes): List<AudioDeviceInfo>? =
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
getAudioDevicesForAttributes(attributes)
} else {
getDevices(AudioManager.GET_DEVICES_OUTPUTS).toList()
}
} catch (e: RuntimeException) {
// Some phones throw here.
Log.w(TAG, "Could not find where audio plays", e)
null
}

private const val TAG = "AudioOutputs"
Original file line number Diff line number Diff line change
Expand Up @@ -1043,6 +1043,12 @@ private void speak(String text) {
private final VolumeChangedListener a11yVolumeChangedListener =
new VolumeChangedListener() {

// Backtalk: this listener says the new accessibility volume, so VolumeMonitor doesn't.
@Override
public boolean announcesVolume(int type) {
return type == AudioManager.STREAM_ACCESSIBILITY;
}

@Override
public void onVolumeChanged(int type, int value, int prevValue, int percent) {
if (type != AudioManager.STREAM_ACCESSIBILITY) {
Expand Down
Loading
Loading